The naming tool's metadata editor saves pre-upload overrides to the
override_metadata table (shared ferrero_tracking DB), but until now the
Python upload pipeline never read from it — every edit was being saved
but never applied to DAM. This wires up the consumer side so user edits
land on the uploaded asset.
- database.py: get_override_metadata() / mark_override_applied(),
resilient to a missing override_metadata table on dev DBs
- metadata_extractor_mvp.py: OVERRIDE_FIELD_MAP (mirrors the naming
tool's editor-field → DAM-field-ID map) + _apply_override_fields().
Applied after master/filename/forced/CreativeX values but before
asset_type_overrides so EOL/LTD compliance still wins. Empty editor
values are skipped (leaves inherited value alone). Validity ISO
dates normalised to MM/DD/YYYY for DAM
- a2_to_a3_upload_polling.py: lookup before building the asset rep,
pass override_fields into build_mvp_asset_representation, mark
applied only after confirmed upload success
Override priority: user edit > master metadata > forced defaults >
hardcoded today+365 validity — so the team's per-asset validity
period (e.g. 1 month) now flows through end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folder-only mode deep-copies the asset template with MAIN_LANGUAGES.values=[]
and never repopulated it from language_code, so the DAM rejected -N uploads
(SND/voiceover) with "Cannot set null value for a required field: MAIN_LANGUAGES".
Now mirrors the full-inheritance path's tabular values structure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
OMG's Box automation treats each new live_campaigns_*.csv as a full-list
replacement, so the per-series global CSV introduced 2026-04-30 stomped
the local list whenever a B1→B2 ran. Collapse to one combined CSV
(A-series + B-series) emitted by every handler.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires B-series (global) campaigns into OMG using the same Box
automation as A-series. Mirrors the A1/A4 lifecycle for B1/B4.
- b1_to_b2_download: after B2 status update, mark live=YES status=B2
and upload live_campaigns_global_<ts>.csv to the existing Box folder
(BOX_LIVE_CAMPAIGNS_FOLDER_ID, 352181382858 in PROD). Filename keeps
the live_campaigns_ prefix so the existing OMG automation rule picks
it up.
- b4_box_uploader (new): polls DAM for status B4, marks live=NO, regens
the global CSV. Mirrors a4_box_uploader.
- a4_box_uploader: reads prior status before overwriting; if it was
B-series, regenerate the global CSV instead. b4_box_uploader does the
symmetric A-series fallback. Defensive in case DAM doesn't enforce
type-specific status transitions.
- database: add get_all_live_global_campaigns() (status LIKE 'B%').
Tighten get_all_live_campaigns() to status LIKE 'A%' so any cross-type
rows can't leak into the wrong CSV.
- orchestrator + orchestrator-prod: register B4 Box Uploader at 10min.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- LTD DAM code confirmed by client: licensingtranslationdocument (was placeholder)
- EOL + LTD: IP Rights forced to "No" (was "Yes")
- EOL + LTD: Remove CreativeX URL and score (not applicable to legal asset types)
- EOL: Description forced to "Legal Studio Name"
- Reorder _apply_asset_type_overrides() to run after _update_creativex_fields()
so overrides have true final precedence (Box CreativeX was clobbering removals)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extracts CreativeX score and URL from DAM master metadata during the
B1→B2 download, persists to creativex_scores with new status
'b1-master-cx-score' (dedup by tracking_id), and surfaces the score in
the b1_to_b2_complete and b1_to_b2_partial emails — falling back to
"No CreativeX Score" when the master has no score yet. Skipped
already-downloaded assets backfill from full_metadata JSONB on next pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds SDA as a new asset type for License claim translations supporting
the EOL (External Legal Opinion) workflow.
- SDA maps to externallegalopinion in DAM (same as EOL).
- Field overrides match EOL (Agency = "-", Prod Company = "-",
Languages = Global, IP Right = Yes, Licensing = No, validity dates
removed) plus a fixed Description: "Translation of License claim -
For approval purposes only".
- Added asset_type_overrides section to field_mappings_ppr.yaml; it
was missing, so EOL overrides weren't actually applying on PPR.
Both EOL and SDA blocks are now defined for both PPR and PROD.
- _apply_asset_type_overrides now appends a simple string field when
the override targets a field not yet in mvp_fields, so the SDA
description is set even if the filename has no subject_title.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Campaign managers often create the campaign in DAM before assets are
uploaded, so an empty Master Assets folder is the normal pre-asset state
rather than a failure. Stop marking these as permanently failed and stop
emailing on every poll.
- increment_a1_retry() gains mark_failed_at_max param; empty-folder path
passes False so the campaign keeps polling indefinitely until assets
appear (or the DAM status changes).
- Empty-folder branch now skips silently on every poll and sends a single
warning email at poll 20 (~1 hour at the 3-min cadence) so genuinely
stuck campaigns still surface.
- New a1_to_a2_no_assets_warning email template — one-time soft warning,
no permanent-failure language.
- Existing reset_a1_retry() on successful A1→A2 still clears the counter
when assets eventually appear.
- Other folder-error paths (folder not found, etc.) keep the original
3-retry-then-fail behavior.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a campaign is re-opened (status reset to A1/B1 after new files are
added), the tool correctly skips already-downloaded assets but the email
report and CSV previously listed the whole folder as "processed", which
was misleading. Reports now show "Total: 14 (12 previously downloaded,
2 new this run)" with new assets in full detail and previously-downloaded
assets in a compact list. B1→B2 CSV gains a Status column matching A1→A2.
DAM subfolder "WND_PCS 2026 2.0" was being treated as a file because
".0" was not in the known extensions list and defaulted to is_folder=False.
This caused an HTTP 404 on download since it's a folder, not a file.
Added numeric-only extension check (.0, .1, etc.) to the folder detection
logic so the script correctly recurses into versioned subfolders and
downloads the assets inside them.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds EOL as a new asset type with field overrides for both PPR and PROD:
- Asset type maps to 'externallegalopinion' in DAM
- Agency Name = "-", Production House = "-"
- Main Languages = "Global"
- IP Rights = "Yes", Licensing = "No"
- Validity dates removed
Also adds VOD platform code and removes OLV asset type.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Folder-only mode (-N suffix files) was sending minimal metadata that DAM
rejected with "unmarshalling parameter" error. Now uses a reference
asset_representation_template.json as the base for all metadata fields,
ensuring the full field structure (column_name, data_type, domain_id, etc.)
the DAM API requires. Also fixes default/forced value handling to use
DomainValue format for domained fields from the template.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Folder-only mode (-N suffix files) was sending simplified metadata that
PROD DAM rejected with "unmarshalling parameter" error. Updated to use
DomainValue format for domained fields, correct asset type field ID
(FERRERO.FIELD.MKTG.ASSET TYPE), asset type code mapping (e.g. SND→sound),
validity dates, and forced values from config.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
A1→A2 now looks up the opentext_id in master_assets for an M-prefixed record
from B1→B2 and stores it as global_master_tracking_id on the local asset record.
This provides traceability from local campaign assets back to their global master
without changing any existing workflow logic or DAM metadata.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously, re-scored assets with a DAM timestamp suffix (e.g. _2026-03-13-05-53-36)
were treated as new files, leaving multiple 'active' records. Now strips the timestamp
and uses LIKE matching so all variants of the same base asset are properly superseded.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mailgun silently drops emails with multiple recipients in the to field.
Send individual API calls per recipient and split comma-separated addresses.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mailgun API is used when MAILGUN_API_KEY and MAILGUN_DOMAIN are set,
with SMTP as fallback for PPR. Also fixes A2→A3 batch subject line
that was rendering Jinja2 syntax literally instead of substituting values.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Was generating FERRERO.TABULAR.FIELD.MARKETING_TAG (underscore) but DAM
expects FERRERO.TABULAR.FIELD.MARKETING.TAG (dot). Added explicit mapping
for tabular field parent table IDs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
VIDEO_POST_PROD_COMPANY and AUDIO_POST_PROD_COMPANY are not domain fields
but were being wrapped with DomainValue, causing unmarshalling errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The _build_fields_from_filename method was using {"value": "..."} without
the required {"type": "string", "value": "..."} structure, causing
unmarshalling errors on the DAM API for -N suffix uploads.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fixes unmarshalling error on DAM upload when using -N suffix files. The API
requires the DomainValue structure when domain_value is true.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The filename_updates logic was only updating field['value'] (singular) but for
tabular fields like MAIN_LANGUAGES, the DAM reads from field['values'] (plural
array). This caused the master's original language (e.g. "Global") to persist
instead of the correct language from the filename (e.g. "PL").
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When no CreativeX score is found for a file, the system was sending a
generic placeholder URL (app.creativex.com/preflight/pretests) to the DAM.
Now sends no URL at all, so only files with actual CreativeX scores get a URL.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove PPR-only gates so PROD supports the same MASTERASSETIDS tabular
field and multi-master ID parsing as PPR. DAM deployment scheduled for
Feb 18 — do not push until then.
Changes:
- filename_parser: Remove is_ppr check, allow multi-master ID parsing in PROD
- a2_to_a3: Populate master_opentext_ids for single-master PROD case
- dam_client: Remove PPR-only skip on domain registration
- metadata_extractor_mvp: Update docstrings only
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CreativeX lookup now falls back to tracking ID search when filename match fails
(handles mismatched naming from CreativeX PDFs). strip_upload_components now
only removes job number and tracking ID, keeping social media codes (YTA, DV3,
etc.) in the clean filename. Updated SOCIAL_MEDIA_CODES from 4 to 39 codes
sourced from the Ferrero naming tool.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The PROD a2_to_a3 script referenced master_opentext_ids without defining it,
causing NameError for all file uploads. Brings in multi-master tracking ID
support from PPR: filename parser handles multiple IDs (PPR) or single ID
(PROD), metadata extractor supports MASTERASSETIDS tabular field.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
A1→A2 now handles re-processing when campaign is reset to A1 after adding new
master assets. Existing assets reuse tracking IDs and skip Box upload, new assets
are processed normally. Also includes PPR domain registration for multiple master
asset IDs in a2_to_a3 and dam_client.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace exposed database credentials and SQL commands in A1 permanently failed notification email with support contact information (optical@oliver.agency).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Remove folder creation logic in get_or_create_subfolder_path() since DAM does not allow folder creation via API. When a subfolder doesn't exist, upload to the parent folder instead of attempting to create it (which was causing 120 second timeouts).
This resolves upload failures in PROD environment during A2→A3 workflow.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit includes critical updates for PPR deployment:
1. Environment-Specific Field Mappings:
- Created field_mappings_ppr.yaml with agency code "Oliver"
- Created field_mappings_prod.yaml with agency code "0000221659"
- Updated config_loader.py to auto-detect environment based on DAM URL
- Enables seamless deployment between PPR and PROD environments
2. Metadata Extractor Enhancements:
- Added MetadataTable extraction support for nested fields
- Enables extraction of "Type of Video & Static Right" multi-value field
- Added logic to apply defaults to existing but empty fields
- Fixed agency name display_value handling for domain fields
3. Default Values Added:
- VIDEO_POST_PROD_COMPANY: "Oliver Marketing Ltd"
- AUDIO_POST_PROD_COMPANY: "Oliver Marketing Ltd"
- PROD_COMPANY (Production House): "-"
These changes ensure:
- Correct agency codes per environment (PPR/PROD)
- Proper extraction of nested tabular fields
- Default values for empty production company fields
- Seamless deployment workflow
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Key Changes:
- Updated metadata_extractor_mvp.py to use SIMPLE structure for all tabular fields
- All tabular fields now use direct value objects (no MetadataTableFieldRow wrapper)
- MAIN_LANGUAGES, ASSETCOMPLIANCE, MARKETING_TAG, CREATIVEX all use SIMPLE structure
- Master Asset ID field updated to SIMPLE structure
- Date fields now use type 'string' instead of 'long'
- Matches DAM reference structure from asset_representation.json
Added Files:
- metadata_extractor_mvp_PROD.py: PROD-specific version with same SIMPLE structure
- Backup files for safety
- Analysis and comparison documentation
Environment:
- Tested and working in PPR environment (ppr.dam.ferrero.com)
- All tabular fields match DAM-supplied reference structure
- Successful uploads confirmed
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Critical Fix:
- extract_global_campaign_reference() now accepts campaign_id parameter
- Always sets local_campaign_id to current campaign as fallback
- Prevents NULL local_campaign_id when no Global Campaign Reference exists
Root Cause:
- Assets without Global Campaign Reference had NULL local_campaign_id
- Caused derivatives to be linked to wrong campaigns
- Same asset in multiple campaigns would share tracking IDs incorrectly
Impact:
- Every asset now has proper local_campaign_id
- Derivatives correctly linked to their source campaign
- Fixes issue where C000001177 assets were showing as C000002098
Changes:
- database.py: Added campaign_id parameter with fallback logic
- a1_to_a2_box_uploader.py: Pass campaign_number to function
- a5_to_a6_download.py: Pass campaign_number to function
Reverted master_asset_id changes per user feedback:
- tracking_id already links derivatives to masters
- No need for additional master_asset_id foreign key
- Only storing dam_asset_id for DAM asset tracking
Changes:
- Reverted get_master_asset() to not return database 'id'
- Updated store_derivative_asset() to only INSERT dam_asset_id
- Updated a2_to_a3_upload_polling.py to pass None for master_asset_id
- Removed master_asset_id from INSERT statement
Note: Migration script still needed for dam_asset_id column only
Email Template Fix:
- Fixed subject line syntax error in a2_to_a3_batch_complete template
- Removed Jinja2 control flow ({% if %}) from subject line
- Changed to simple expression-only format
- Fixes 'Failed to send email' error
Database Logging Fix:
- Updated get_master_asset() to return database primary key 'id'
- Updated store_derivative_asset() to actually store master_asset_id and dam_asset_id
- Updated a2_to_a3_upload_polling.py to pass master_asset['id'] instead of None
- Added migration script to add dam_asset_id column to derivative_assets table
- Fixes issue where derivatives weren't being linked to masters in database
- Enables proper lookups and tracking of uploaded derivatives
Impact:
- Email notifications will now send successfully
- Derivatives will be properly logged and linked to master assets
- Other tools can now find uploaded derivatives in database
- Convert dates to milliseconds since epoch (Unix timestamp × 1000)
- Change field type from 'string' to 'long' for DATE fields
- Add _set_date_field_value() helper method for proper date handling
- Fixes 'java.lang.String was specified. Expecting java.util.Date' error
- Applies to A2->A3 uploads
