Commit graph

120 commits

Author SHA1 Message Date
DJP
499399d6c6 Sanitize subfolder names in dam_client recursive search 2025-12-01 15:16:55 -05:00
DJP
2a8129f428 Fix Box folder naming: sanitize forward slashes in campaign names 2025-12-01 15:13:58 -05:00
DJP
4d1e71d978 Implement true recursion in DAM client: treat items without extensions as folders 2025-12-01 14:54:21 -05:00
DJP
67c8765512 Add fallback check to B1->B2 script: skip assets with no file extension 2025-12-01 14:22:59 -05:00
DJP
0f6e816255 Fix recursive download in B1->B2 script: improve folder identification and add safeguards 2025-12-01 14:17:14 -05:00
DJP
4c691dda30 Update CSV format to combine Campaign Number and Name, add manual regeneration script 2025-12-01 09:22:32 -05:00
DJP
db0dceb19c Add --auth-pfx-v2 and --env options to update_campaign_status.py 2025-11-27 06:58:39 -05:00
DJP
4259b1073c Add --auth-pfx-v2 support to A4 Box Uploader 2025-11-26 17:01:34 -05:00
DJP
17648d9051 Fix test_connection to not use non-existent /v6 endpoint 2025-11-26 16:58:55 -05:00
DJP
532699f634 Show full OAuth tokens in logs for debugging 2025-11-26 16:44:07 -05:00
DJP
cda33e8c8d Switch production orchestrator back to mTLS V2 (--auth-pfx-v2) 2025-11-26 16:32:43 -05:00
DJP
b2c8e01d4b Add token preview logging for OAuth tokens 2025-11-26 16:32:10 -05:00
DJP
c7ddf1d45f Add verbose logging for all DAM API requests 2025-11-26 16:26:15 -05:00
DJP
d68e149d1e Switch production orchestrator to use legacy mTLS (--auth-pfx) 2025-11-26 16:05:10 -05:00
DJP
37cab9d902 Add support for loading environment-specific .env files (prod/dev) 2025-11-26 15:44:09 -05:00
DJP
c1bd575bf9 Add --auth-pfx-v2 support to A4 Webhook Monitor for production 2025-11-26 15:39:00 -05:00
DJP
c1df6a7712 Fix A4 Webhook Monitor to use --auth-pfx for production 2025-11-26 15:38:06 -05:00
DJP
87b593c5f9 Fix syntax errors in orchestrator-prod.py args 2025-11-26 15:34:14 -05:00
DJP
c53e79cbaf Add production orchestrator configuration 2025-11-26 15:31:32 -05:00
DJP
07bce09d65 Fix B1→B2 bug: total_assets referenced before assignment 2025-11-26 15:07:27 -05:00
DJP
8ca44fcf1e Add metadata diagnostic tool for troubleshooting field issues 2025-11-26 14:52:39 -05:00
DJP
f9c11ef3f5 Fix misleading log message for A5 campaigns with no rejections 2025-11-26 14:45:48 -05:00
DJP
795e4e7d96 Improve A5 notification logic to handle status changes 2025-11-26 14:26:36 -05:00
DJP
0f1c3dd0ec Prevent duplicate 'no rejections' emails for A5 campaigns 2025-11-26 14:25:57 -05:00
DJP
936071d7ad Change script intervals from 5 to 3 minutes for faster processing 2025-11-26 14:18:45 -05:00
DJP
f15ae9a8d1 Stream full script output to console in real-time 2025-11-26 14:04:15 -05:00
DJP
d6b68af5d5 Fix A5→A6 to also use OAuth authentication 2025-11-26 14:03:07 -05:00
DJP
16527f6e43 Temporarily disable mTLS auth, use OAuth for troubleshooting 2025-11-26 13:59:48 -05:00
DJP
3518f7c909 Remove A1->A2 Download task from orchestrator and add run guide 2025-11-26 13:53:48 -05:00
DJP
7599fe7cd2 feat: Remove A1->A2 Download script from orchestrator configuration 2025-11-26 13:48:48 -05:00
DJP
99d8621266 Increase throughput: process 2 campaigns in A1→A2, all files in A2→A3 2025-11-26 13:43:06 -05:00
DJP
98fb7eaee2 Fix smoke test to use prod-auth endpoint instead of test-auth 2025-11-26 10:20:59 -05:00
DJP
9d207d0480 chore: Update DAM mTLS base and OAuth URLs in production environment to /token endpoint. 2025-11-26 10:18:56 -05:00
DJP
6064b0971e Fix smoke test to explicitly load .env-prod file 2025-11-26 10:14:26 -05:00
DJP
cabc1d5548 Add production smoke test script for mTLS V2 authentication 2025-11-26 10:08:04 -05:00
DJP
c1f338022c fix: Ensure type field is added when updating CreativeX URL
- Modified _set_field_value to include 'type': 'string' in all code paths
- Adds type field when updating existing CreativeX URL field
- Ensures consistent structure whether creating or updating field
2025-11-25 09:14:25 -05:00
DJP
80316cad32 fix: Add missing type field to CreativeX URL metadata
- Added 'type': 'string' to FERRERO.FIELD.CREATIVEX LINK value structure
- Fixes DAM validation error for CreativeX URL field
- Structure now matches DAM requirements
2025-11-25 09:11:41 -05:00
DJP
548c30344b feat: Support multiple CreativeX platforms in metadata
- Updated creativex_scoring_storing.py to map multiple placements to platforms
- Modified get_mapped_platform to get_mapped_platforms (returns list)
- Updated a2_to_a3_upload_polling.py to retrieve platforms list from DB
- Enhanced metadata_extractor_mvp.py to build multi-value CreativeX field
- Added DAM-CX mappings.csv for channel/placement to platform mapping
- Supports single channel with multiple placements generating multiple Platform^Score values
2025-11-24 14:44:11 -05:00
DJP
491fc8e938 feat: Add A1→A3 campaign advance script, introduce systemd service for orchestrator, and ref 2025-11-24 13:50:16 -05:00
DJP
0af15563bc feat: Implement new Python script locking, relocate PHP workflow, and update Python scripts and documentation. 2025-11-21 17:20:34 -05:00
DJP
22069ed66f refactor: Relocate test scripts to a dedicated tests/ directory and introduce orchestrator.py. 2025-11-21 17:10:04 -05:00
DJP
2cad2c2955 Fix AttributeError in DAMClient.test_connection 2025-11-21 16:57:43 -05:00
DJP
5aeab8d9a3 Update a1_to_a2_download.py to support Auth V2 2025-11-21 16:53:32 -05:00
DJP
6fe2ba234b Implement Auth V2 (Hybrid mTLS/OAuth) and update field mappings 2025-11-21 16:46:37 -05:00
DJP
b906434f67 Add A1->A2 and A4 Box CSV uploader scripts 2025-11-20 22:52:26 -05:00
DJP
20a187a61d feat: Add Python project dependencies. 2025-11-20 22:45:04 -05:00
DJP
0bf3c34cd0 Add asset type mapping from 3-letter codes to DAM codes
Maps frontend naming tool 3-letter codes (EHI, IMG, TVC) to DAM's
lowercase descriptive codes (heroimage, keyvisual, tvc) for uploads.

New File: config/asset_type_mappings.yaml
- 45 asset type mappings from DAM lookup domains
- Maps 3-letter codes to DAM codes
- E-Commerce types: EHI→heroimage, EBS→beautyshot, etc.
- Standard types: IMG→keyvisual, TVC→tvc, etc.
- Expandable as new asset types added

Field Mappings Update (field_mappings.yaml):
- Added FERRERO.FIELD.MKTG.ASSET TYPE to filename_updates
- Source: asset_type (from parsed filename)
- Required: true

Metadata Extractor Updates (metadata_extractor_mvp.py):
- Added _load_asset_type_mappings() method
- Added _map_asset_type() method
- Integrated mapping into _update_fields()
- Logs mappings: "Asset type mapping: EHI -> heroimage"
- Warns if no mapping found (may fail DAM validation)

Example Flow:
1. Filename: ROC_TEST-E2E2_EHI_1x1_DE_de.png
2. Parse: asset_type = "EHI"
3. Map: EHI → "heroimage"
4. Update field: FERRERO.FIELD.MKTG.ASSET TYPE = "heroimage"
5. DAM accepts "heroimage" (valid domain value)

Without Mapping (before):
- Field value: "EHI"
- DAM validation: FAILS (EHI not in domain)

With Mapping (after):
- Field value: "heroimage"
- DAM validation: PASSES 

Complete Mapping List (45 types):
E-Commerce: ECA, ECB, EBS, EBR, EEM, EHI, EIL, EOP, EUG, EWB
Standard: 3RT, APC, BBK, BRC, BSG, CKV, CID, DAT, FLA, FNT, GDT,
         GRG, IMG, FPO, LGL, LOG, MLF, OLV, PAW, PKI, POS, PDM,
         PRI, QRC, SND, SIP, SGL, TVC, VIE

DAM Answer: Uses descriptive lowercase codes, not 3-letter codes.
Frontend uses 3-letter for brevity, backend maps to DAM format.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 13:08:01 -05:00
DJP
d1ab8551e5 Add asset type filename update and make field updates configurable
Enables asset type to be updated from derivative filename and refactors
_update_fields() to use filename_updates configuration dynamically.

Field Mappings Configuration (field_mappings.yaml):

Added to filename_updates:
- FERRERO.FIELD.MKTG.ASSET TYPE:
    source: asset_type
    required: true

Now updates from derivative filename:
- ROC_TEST-E2E2_EHI_1x1_DE_de.png → Asset Type = "EHI"

Metadata Extractor Refactor (metadata_extractor_mvp.py):

Old _update_fields():
- Hardcoded field updates (ASSET NAME, DESCRIPTION, STATE)
- Not using filename_updates configuration
- Required code changes to add new fields

New _update_fields():
- Dynamically processes filename_updates from config
- Supports transform: uppercase/lowercase
- Supports any source field from parsed_filename
- Uses forced_values from config (was hardcoded before)
- Add new fields via config, no code changes needed

Configuration-Driven Updates:
- ARTESIA.FIELD.ASSET NAME ← clean_filename
- ARTESIA.FIELD.ASSET DESCRIPTION ← subject_title
- FERRERO.FIELD.MKTG.ASSET TYPE ← asset_type (NEW)
- MAIN_LANGUAGES ← language_code (uppercase)
- FERRERO.FIELD.STATE ← "Local" (forced value)

Benefits:
- Asset type now correctly populated from filename
- Configuration-driven (add fields without code changes)
- Cleaner code (uses config instead of hardcoded logic)
- Forced values also configurable
- Easier to maintain and extend

Example:
Filename: ROC_TEST-E2E2_EHI_1x1_DE_de.png
Parsed asset_type: "EHI"
Field FERRERO.FIELD.MKTG.ASSET TYPE updated to: "EHI"

Impact:
All A2→A3 uploads will now have correct Asset Type from derivative
filename instead of inheriting from master (which may be different).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 12:58:10 -05:00
DJP
e44f42dad5 Add master CreativeX score extraction and storage in A1→A2 workflow
Stores master asset CreativeX scores from DAM metadata during A1→A2
download for reference/reporting purposes (not used in uploads).

Database Changes:

creativex_scores table:
- Added: tracking_id VARCHAR(6) column
- Added: idx_creativex_tracking_id index
- Updated comment: status can be 'active', 'superseded', or 'master-cx-score'

Status Values:
- 'active' - Current derivative score (from PDF extraction)
- 'superseded' - Old derivative score (version history)
- 'master-cx-score' - Master asset score (from A1→A2 DAM metadata) ← NEW

Migration SQL (for existing databases):
ALTER TABLE creativex_scores ADD COLUMN tracking_id VARCHAR(6);
CREATE INDEX idx_creativex_tracking_id ON creativex_scores(tracking_id);

Database Method Updates (database.py):

store_creativex_score() signature:
- Added: tracking_id parameter (optional, default None)
- Added: status parameter (optional, default 'active')

Logic:
- If status='master-cx-score': Simple insert, no versioning
- If status='active': Soft delete versioning as before
- Always stores tracking_id if provided

A1→A2 Script Updates (a1_to_a2_download.py):

New Function: extract_creativex_from_dam_metadata()
- Searches metadata_element_list for CREATIVEX fields
- Extracts FERRERO.TAB.FIELD.CREATIVEX (score)
- Extracts FERRERO.FIELD.CREATIVEX LINK (url)
- Returns dict with score/url or None if not found
- Handles tabular field structure for score
- Handles nested value structure for URL

Integration:
- After successful master asset storage
- Extracts CreativeX from asset metadata
- If found: Stores in creativex_scores with status='master-cx-score'
- Links to master via tracking_id
- Logs when score found/stored
- Logs "normal" when not found (not all masters are scored)

Use Cases:

A2→A3 Upload:
- Still uses filename-based lookup ONLY 
- No changes to A2→A3 logic 
- Master scores not used for uploads 

Reporting/Analytics Tools:
- Can query master score by tracking_id
- Compare master vs derivative scores
- Track score improvements
- Audit trail

Query Examples:
-- Get master score for tracking ID
SELECT * FROM creativex_scores
WHERE tracking_id = '7xXgKp' AND status = 'master-cx-score';

-- Get derivative score for filename
SELECT * FROM creativex_scores
WHERE filename = 'file.mp4' AND status = 'active';

Test Record Created:
- Filename: nutella_pbased.jpg
- Tracking ID: 7xXgKp
- Score: 85
- Status: master-cx-score

Benefits:
- Historical reference of master scores
- Enables score comparison analytics
- No impact on A2→A3 upload logic
- Automatic extraction during A1→A2
- Optional (works even if masters don't have scores)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 13:56:22 -05:00
DJP
68180b23cf Update filename parser to V2.1 structure with new field positions
Complete rewrite of filename parser to support new field order where
Subject/Asset moved up and Country/Language moved down, plus new
Social Media field.

BREAKING CHANGE: V2.1 Structure (November 2025)

Old (V1):
[JOB]_[BRAND]_[COUNTRY]_[LANG]_[SUBJECT]_[ASSET]_[SPOT]_[DUR]_[RATIO]_[TRACKING]

New (V2.1):
[JOB]_[BRAND]_[SUBJECT]_[ASSET]_[DUR]_[RATIO]_[SPOT]_[COUNTRY]_[LANG]_[SOCIAL]_[TRACKING]

Field Position Changes:
- Subject Title: Position 5 → 3 (MOVED UP)
- Asset Type: Position 6 → 4 (MOVED UP)
- Duration: Position 8 → 5 (MOVED UP)
- Aspect Ratio: Position 9 → 6 (MOVED UP)
- Spot Version: 7 → 7 (SAME)
- Country Code: Position 3 → 8 (MOVED DOWN)
- Language Code: Position 4 → 9 (MOVED DOWN)
- Social Media: NEW → Position 10
- Tracking ID: Position 10 → 11

New Social Media Field:
- Field: social_media_version
- Position: 10 (after language, before tracking)
- Format: 3 uppercase letters
- Codes: FBP, FBR, IGF, IGR (expandable)
- Optional: Only present for social media assets

Parse Algorithm Changes:
- Positions 1-4 now: Job, Brand, Subject, Asset (fixed)
- Positions 5-11: Pattern-based detection (flexible)
- Duration detected by \d+S pattern
- Aspect ratio detected by \d+x\d+ or contains 'x'
- Spot detected by MST/REF
- Country detected by 2 upper alpha (after ratio)
- Language detected by 2-3 lower alpha (after country)
- Social detected by known codes (after language)
- Tracking ID detected by 6 alphanumeric + optional -N

strip_upload_components() Updated:
Now outputs: [BRAND]_[SUBJECT]_[ASSET]_[DUR]_[RATIO]_[SPOT]_[COUNTRY]_[LANG]_[SOCIAL]
- Includes social media version if present
- Still strips job number and tracking ID

Testing:
All 7 test cases from specification passed:
 All fields present
 Minimal (no duration/social/tracking)
 No duration
 No spot version
 With -N tracking (folder-only mode)
 No social media (most common)
 No tracking ID

Example:
Input:  1234567_RAF_TEST_OLV_6S_1x1_REF_GL_it_IGF_abc123.mp4
Parsed: brand=RAF, subject=TEST, country=GL, lang=it, social=IGF
Clean:  RAF_TEST_OLV_6S_1x1_REF_GL_it_IGF.mp4

Backward Compatibility:
None - system not live yet, clean cutover to V2.1 format only.

Backup: filename_parser_v1_backup.py contains old version for reference.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 13:41:19 -05:00