The naming tool's metadata editor saves pre-upload overrides to the
override_metadata table (shared ferrero_tracking DB), but until now the
Python upload pipeline never read from it — every edit was being saved
but never applied to DAM. This wires up the consumer side so user edits
land on the uploaded asset.
- database.py: get_override_metadata() / mark_override_applied(),
resilient to a missing override_metadata table on dev DBs
- metadata_extractor_mvp.py: OVERRIDE_FIELD_MAP (mirrors the naming
tool's editor-field → DAM-field-ID map) + _apply_override_fields().
Applied after master/filename/forced/CreativeX values but before
asset_type_overrides so EOL/LTD compliance still wins. Empty editor
values are skipped (leaves inherited value alone). Validity ISO
dates normalised to MM/DD/YYYY for DAM
- a2_to_a3_upload_polling.py: lookup before building the asset rep,
pass override_fields into build_mvp_asset_representation, mark
applied only after confirmed upload success
Override priority: user edit > master metadata > forced defaults >
hardcoded today+365 validity — so the team's per-asset validity
period (e.g. 1 month) now flows through end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wraps find_campaign_by_identifier() from update_campaign_status.py so
operators can query a campaign's current DAM status by number or partial
name without performing any updates.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folder-only mode deep-copies the asset template with MAIN_LANGUAGES.values=[]
and never repopulated it from language_code, so the DAM rejected -N uploads
(SND/voiceover) with "Cannot set null value for a required field: MAIN_LANGUAGES".
Now mirrors the full-inheritance path's tabular values structure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
OMG's Box automation treats each new live_campaigns_*.csv as a full-list
replacement, so the per-series global CSV introduced 2026-04-30 stomped
the local list whenever a B1→B2 ran. Collapse to one combined CSV
(A-series + B-series) emitted by every handler.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A1_RETRY_LOGIC.md updated to reflect the 2026-04-28 rework: empty
folders are now treated as expected workflow (silent skip + one-time
warning at poll 20, no auto permanent-fail), while the original
3-strikes-then-permanently-fail behavior is preserved for genuine
folder errors via the mark_failed_at_max flag.
README.md adds LTD (Licensing Translation Document) to the asset type
override section alongside EOL, and notes that empty overrides remove
fields while non-empty overrides on non-MVP fields are appended.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires B-series (global) campaigns into OMG using the same Box
automation as A-series. Mirrors the A1/A4 lifecycle for B1/B4.
- b1_to_b2_download: after B2 status update, mark live=YES status=B2
and upload live_campaigns_global_<ts>.csv to the existing Box folder
(BOX_LIVE_CAMPAIGNS_FOLDER_ID, 352181382858 in PROD). Filename keeps
the live_campaigns_ prefix so the existing OMG automation rule picks
it up.
- b4_box_uploader (new): polls DAM for status B4, marks live=NO, regens
the global CSV. Mirrors a4_box_uploader.
- a4_box_uploader: reads prior status before overwriting; if it was
B-series, regenerate the global CSV instead. b4_box_uploader does the
symmetric A-series fallback. Defensive in case DAM doesn't enforce
type-specific status transitions.
- database: add get_all_live_global_campaigns() (status LIKE 'B%').
Tighten get_all_live_campaigns() to status LIKE 'A%' so any cross-type
rows can't leak into the wrong CSV.
- orchestrator + orchestrator-prod: register B4 Box Uploader at 10min.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- LTD DAM code confirmed by client: licensingtranslationdocument (was placeholder)
- EOL + LTD: IP Rights forced to "No" (was "Yes")
- EOL + LTD: Remove CreativeX URL and score (not applicable to legal asset types)
- EOL: Description forced to "Legal Studio Name"
- Reorder _apply_asset_type_overrides() to run after _update_creativex_fields()
so overrides have true final precedence (Box CreativeX was clobbering removals)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DAM stores the CreativeX tabular cell as '<platform>^<score>', e.g.
'DV360^100'. Add format_cx_score_for_display() and apply at the point
where the email asset dict is built — both new-download and skipped
paths. Raw value stays in creativex_scores.quality_score so all platform
info is preserved for queries; only the email display is reshaped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Diagnostic confirmed FERRERO.TAB.FIELD.CREATIVEX (score) lives at depth 2
in B1 master metadata — nested under FERRERO.TABULAR.FIELD.CREATIVEX
inside a category — and FERRERO.FIELD.CREATIVEX LINK lives at depth 1.
The flat top-level walk used previously never reached them, so live B1
runs and the backfill both reported zero CX scores. Updated extractor
in b1_to_b2_download.py and the inline copy in
backfill_b1_creativex_scores.py to descend recursively.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous version only looked at top-level metadata_element_list, which
contains categories — actual fields nest under each category. Now
recursively descends through all nested metadata_element_list arrays
and counts every element_id at any depth, then searches the full set
for CX/score/quality hints. Reports max nesting depth and the depth at
which each CX-flavored ID was found.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
psycopg2 performs %-substitution when params are passed to execute(),
so 'M%' in the LIKE clause was being interpreted as a positional
placeholder, raising IndexError when there's only one real %s (LIMIT).
Escape as 'M%%' so it's preserved as a literal percent.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Read-only script that samples B1 global masters from master_assets and
reports: top-level keys in full_metadata, presence of
metadata.metadata_element_list, and any element_ids matching
creativex/cx/score/quality (case-insensitive). Helps diagnose why the CX
backfill found 0 matches — distinguishes "client masters have no CX
score yet" from "CX field uses a different element_id than A1".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Walks master_assets for B1 global masters (tracking_id LIKE 'M%' AND
local_campaign_id IS NULL), extracts CreativeX score from full_metadata
JSONB, and inserts into creativex_scores with status='b1-master-cx-score'.
Idempotent — relies on the existing tracking_id dedup in
db.store_creativex_score, so re-runs are safe. Supports --dry-run for
preview before applying.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extracts CreativeX score and URL from DAM master metadata during the
B1→B2 download, persists to creativex_scores with new status
'b1-master-cx-score' (dedup by tracking_id), and surfaces the score in
the b1_to_b2_complete and b1_to_b2_partial emails — falling back to
"No CreativeX Score" when the master has no score yet. Skipped
already-downloaded assets backfill from full_metadata JSONB on next pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Renames the asset type code introduced in 0f49cc6 from SDA (Supporting
Documents for Approval) to LTD (Licensing Translation Document). All
field overrides and the fixed Description value are unchanged.
DAM-side asset type code remains externallegalopinion as a placeholder
pending client confirmation; will update in a follow-up commit if the
DAM code differs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds SDA as a new asset type for License claim translations supporting
the EOL (External Legal Opinion) workflow.
- SDA maps to externallegalopinion in DAM (same as EOL).
- Field overrides match EOL (Agency = "-", Prod Company = "-",
Languages = Global, IP Right = Yes, Licensing = No, validity dates
removed) plus a fixed Description: "Translation of License claim -
For approval purposes only".
- Added asset_type_overrides section to field_mappings_ppr.yaml; it
was missing, so EOL overrides weren't actually applying on PPR.
Both EOL and SDA blocks are now defined for both PPR and PROD.
- _apply_asset_type_overrides now appends a simple string field when
the override targets a field not yet in mvp_fields, so the SDA
description is set even if the filename has no subject_title.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Campaign managers often create the campaign in DAM before assets are
uploaded, so an empty Master Assets folder is the normal pre-asset state
rather than a failure. Stop marking these as permanently failed and stop
emailing on every poll.
- increment_a1_retry() gains mark_failed_at_max param; empty-folder path
passes False so the campaign keeps polling indefinitely until assets
appear (or the DAM status changes).
- Empty-folder branch now skips silently on every poll and sends a single
warning email at poll 20 (~1 hour at the 3-min cadence) so genuinely
stuck campaigns still surface.
- New a1_to_a2_no_assets_warning email template — one-time soft warning,
no permanent-failure language.
- Existing reset_a1_retry() on successful A1→A2 still clears the counter
when assets eventually appear.
- Other folder-error paths (folder not found, etc.) keep the original
3-retry-then-fail behavior.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The A1→A2 uploader processes up to 2 campaigns per run. Permanently-failed
campaigns were skipped only inside the loop, so they still consumed slots
and could starve the rest of the queue indefinitely. Filter them out
before the slice so eligible campaigns get processed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a campaign is re-opened (status reset to A1/B1 after new files are
added), the tool correctly skips already-downloaded assets but the email
report and CSV previously listed the whole folder as "processed", which
was misleading. Reports now show "Total: 14 (12 previously downloaded,
2 new this run)" with new assets in full detail and previously-downloaded
assets in a compact list. B1→B2 CSV gains a Status column matching A1→A2.
Remove 9 deprecated types (CID, ECB, EBS, EOP, EUG, EWB, FPO, PKI, PRI),
add 9 new types (EAN, ESI, NTB, PIR, PKC, PKT, SCP, SNC, UPI), and update
DAT DAM code from digitalassettoolkit to digitalasset. Display names updated
to match current client naming conventions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DAM subfolder "WND_PCS 2026 2.0" was being treated as a file because
".0" was not in the known extensions list and defaulted to is_folder=False.
This caused an HTTP 404 on download since it's a folder, not a file.
Added numeric-only extension check (.0, .1, etc.) to the folder detection
logic so the script correctly recurses into versioned subfolders and
downloads the assets inside them.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DAM subfolder "WND_PCS 2026 2.0" was being treated as a downloadable
asset because ".0" passed the existing extension check. Added safeguard
to skip items with numeric-only extensions (e.g. .0, .1) which are
version numbers in folder names, not real files.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously the script re-downloaded and re-uploaded all assets on every
retry, even those already successfully stored in DB and Box. For large
campaigns (1300+ assets) this caused unnecessary load and duplicate uploads.
Now checks DB via find_global_master_by_opentext_id() before downloading.
Assets already in DB with a valid Box URL are skipped and counted toward
the processed total, so only genuinely failed assets are retried.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Added documentation for template-based folder-only mode (-N flag),
asset type overrides (EOL), environment-specific field mappings,
and updated config file references.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds EOL as a new asset type with field overrides for both PPR and PROD:
- Asset type maps to 'externallegalopinion' in DAM
- Agency Name = "-", Production House = "-"
- Main Languages = "Global"
- IP Rights = "Yes", Licensing = "No"
- Validity dates removed
Also adds VOD platform code and removes OLV asset type.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Folder-only mode (-N suffix files) was sending minimal metadata that DAM
rejected with "unmarshalling parameter" error. Now uses a reference
asset_representation_template.json as the base for all metadata fields,
ensuring the full field structure (column_name, data_type, domain_id, etc.)
the DAM API requires. Also fixes default/forced value handling to use
DomainValue format for domained fields from the template.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Folder-only mode (-N suffix files) was sending simplified metadata that
PROD DAM rejected with "unmarshalling parameter" error. Updated to use
DomainValue format for domained fields, correct asset type field ID
(FERRERO.FIELD.MKTG.ASSET TYPE), asset type code mapping (e.g. SND→sound),
validity dates, and forced values from config.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
A1→A2 now looks up the opentext_id in master_assets for an M-prefixed record
from B1→B2 and stores it as global_master_tracking_id on the local asset record.
This provides traceability from local campaign assets back to their global master
without changing any existing workflow logic or DAM metadata.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously, re-scored assets with a DAM timestamp suffix (e.g. _2026-03-13-05-53-36)
were treated as new files, leaving multiple 'active' records. Now strips the timestamp
and uses LIKE matching so all variants of the same base asset are properly superseded.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mailgun silently drops emails with multiple recipients in the to field.
Send individual API calls per recipient and split comma-separated addresses.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Diagnose daily report email delivery issue - tests single recipient,
comma-separated string in list, and properly split list formats.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mailgun API is used when MAILGUN_API_KEY and MAILGUN_DOMAIN are set,
with SMTP as fallback for PPR. Also fixes A2→A3 batch subject line
that was rendering Jinja2 syntax literally instead of substituting values.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Was generating FERRERO.TABULAR.FIELD.MARKETING_TAG (underscore) but DAM
expects FERRERO.TABULAR.FIELD.MARKETING.TAG (dot). Added explicit mapping
for tabular field parent table IDs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
VIDEO_POST_PROD_COMPANY and AUDIO_POST_PROD_COMPANY are not domain fields
but were being wrapped with DomainValue, causing unmarshalling errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The _build_fields_from_filename method was using {"value": "..."} without
the required {"type": "string", "value": "..."} structure, causing
unmarshalling errors on the DAM API for -N suffix uploads.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fixes unmarshalling error on DAM upload when using -N suffix files. The API
requires the DomainValue structure when domain_value is true.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The filename_updates logic was only updating field['value'] (singular) but for
tabular fields like MAIN_LANGUAGES, the DAM reads from field['values'] (plural
array). This caused the master's original language (e.g. "Global") to persist
instead of the correct language from the filename (e.g. "PL").
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When no CreativeX score is found for a file, the system was sending a
generic placeholder URL (app.creativex.com/preflight/pretests) to the DAM.
Now sends no URL at all, so only files with actual CreativeX scores get a URL.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove PPR-only gates so PROD supports the same MASTERASSETIDS tabular
field and multi-master ID parsing as PPR. DAM deployment scheduled for
Feb 18 — do not push until then.
Changes:
- filename_parser: Remove is_ppr check, allow multi-master ID parsing in PROD
- a2_to_a3: Populate master_opentext_ids for single-master PROD case
- dam_client: Remove PPR-only skip on domain registration
- metadata_extractor_mvp: Update docstrings only
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The new CreativeX API format stores channel/publisher at the top level
of full_extraction_data instead of inside a data.ferrero_mapped_platforms
wrapper. Add fallback mapping so platforms are correctly populated for
DAM uploads.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
YouTube Ads was missing from the DAM-CX mappings CSV, causing empty
Platform > Rating fields for YouTube assets. Also adds a fallback that
derives the CreativeX platform from the filename social media code (e.g.
YTA -> YouTube) when the database has no mapped platforms.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Populates master_opentext_ids for single-master case so uploads use the
tabular FERRERO.MASTERASSETIDS field instead of the ARTESIA.FIELD.ASSET_ID
fallback. Reverts the workaround from 6517a4f now that the field is being
configured in PROD DAM.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CreativeX lookup now falls back to tracking ID search when filename match fails
(handles mismatched naming from CreativeX PDFs). strip_upload_components now
only removes job number and tracking ID, keeping social media codes (YTA, DV3,
etc.) in the clean filename. Updated SOCIAL_MEDIA_CODES from 4 to 39 codes
sourced from the Ferrero naming tool.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PROD DAM rejects FERRERO.MASTERASSETIDS as it only exists in PPR. Remove the
single-master-to-list conversion so PROD uses the existing single-ID field
(master_opentext_id) instead. Will be re-added when client configures the
tabular field in PROD.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
