Commit graph

142 commits

Author SHA1 Message Date
DJP
2ef92674d3 Feat: Add subfolder path to A2->A3 email notifications 2026-01-19 10:20:51 -05:00
DJP
ee6afe0888 Feat: Configurable Master Asset ID & Fix Date Type 2026-01-19 09:40:55 -05:00
DJP
0f984f3d7b Fix: Update asset validity date format to ISO-8601 string 2026-01-16 13:53:41 -05:00
DJP
631dba4390 Fix campaign ID storage - always set local_campaign_id
Critical Fix:
- extract_global_campaign_reference() now accepts campaign_id parameter
- Always sets local_campaign_id to current campaign as fallback
- Prevents NULL local_campaign_id when no Global Campaign Reference exists

Root Cause:
- Assets without Global Campaign Reference had NULL local_campaign_id
- Caused derivatives to be linked to wrong campaigns
- Same asset in multiple campaigns would share tracking IDs incorrectly

Impact:
- Every asset now has proper local_campaign_id
- Derivatives correctly linked to their source campaign
- Fixes issue where C000001177 assets were showing as C000002098

Changes:
- database.py: Added campaign_id parameter with fallback logic
- a1_to_a2_box_uploader.py: Pass campaign_number to function
- a5_to_a6_download.py: Pass campaign_number to function
2025-12-22 11:37:58 -05:00
DJP
5586dcc5de Simplify derivative storage - only store dam_asset_id
Reverted master_asset_id changes per user feedback:
- tracking_id already links derivatives to masters
- No need for additional master_asset_id foreign key
- Only storing dam_asset_id for DAM asset tracking

Changes:
- Reverted get_master_asset() to not return database 'id'
- Updated store_derivative_asset() to only INSERT dam_asset_id
- Updated a2_to_a3_upload_polling.py to pass None for master_asset_id
- Removed master_asset_id from INSERT statement

Note: Migration script still needed for dam_asset_id column only
2025-12-22 10:16:14 -05:00
DJP
c901a79e24 Fix A2→A3 email template and database logging issues
Email Template Fix:
- Fixed subject line syntax error in a2_to_a3_batch_complete template
- Removed Jinja2 control flow ({% if %}) from subject line
- Changed to simple expression-only format
- Fixes 'Failed to send email' error

Database Logging Fix:
- Updated get_master_asset() to return database primary key 'id'
- Updated store_derivative_asset() to actually store master_asset_id and dam_asset_id
- Updated a2_to_a3_upload_polling.py to pass master_asset['id'] instead of None
- Added migration script to add dam_asset_id column to derivative_assets table
- Fixes issue where derivatives weren't being linked to masters in database
- Enables proper lookups and tracking of uploaded derivatives

Impact:
- Email notifications will now send successfully
- Derivatives will be properly logged and linked to master assets
- Other tools can now find uploaded derivatives in database
2025-12-22 10:12:36 -05:00
DJP
222a53f466 Fix date field type error for ASSET VALIDITY START/END PERIOD
- Convert dates to milliseconds since epoch (Unix timestamp × 1000)
- Change field type from 'string' to 'long' for DATE fields
- Add _set_date_field_value() helper method for proper date handling
- Fixes 'java.lang.String was specified. Expecting java.util.Date' error
- Applies to A2->A3 uploads
2025-12-19 23:02:55 -05:00
DJP
5e8745580a Fix folder detection for folders with periods (v2)
- Enhanced logic to detect false-positive extensions like '. REFERENCE FILES'
- Checks if extension starts with period + space/uppercase (indicates folder name)
- Checks if extension contains spaces (not a valid file extension)
- Properly handles edge cases: '01. REFERENCE FILES', '02. OPEN FILES'
- Tested with multiple scenarios to ensure accuracy
2025-12-17 13:14:59 -05:00
DJP
6d7aa7e95a Fix folder detection for folders with periods in names
- Enhanced _get_assets_recursive() to properly identify folders vs files
- Added resource_type field validation (more reliable than asset_type)
- Created whitelist of 40+ known file extensions for accurate detection
- Fixes HTTP 404 errors when processing folders like '01. REFERENCE FILES'
- Applies to both A1->A2 and B1->B2 workflows
2025-12-17 13:06:02 -05:00
DJP
599d468e44 Add Master Asset ID field to A2→A3 uploads
- Added ARTESIA.FIELD.ASSET_ID to MVP fields in field_mappings.yaml
- Updated metadata_extractor_mvp.py to accept master_opentext_id parameter
- Added _add_master_asset_id_field() and _get_field_id() helper methods
- Modified a2_to_a3_upload_polling.py to pass master asset's opentext_id
- Field is populated with original master asset's DAM ID for derivative tracking
- Field is omitted for new assets (tracking ID with -N suffix)
- Covers both A2→A3 standard derivatives and A5→A6 reworked assets
2025-12-15 17:28:30 -05:00
DJP
022f232224 Fix UnboundLocalError in A1->A2 uploader and add error email 2025-12-10 08:36:16 -05:00
DJP
a7d17dd00d Add CSV attachment with campaign details to A1 and B1 success emails 2025-12-07 19:27:36 -05:00
DJP
9a3bf5d9cd Implement Master Tracking ID validation for A2-A3 uploads 2025-12-06 17:21:38 -05:00
DJP
160ef8ad43 Implement prefix-based tracking ID system for master files
- Master files (B1→B2) now always start with 'M' prefix
- Regular files (A1→A2, A5→A6) never start with 'M'
- Updated generate_unique_tracking_id() to accept is_master parameter
- All tracking IDs remain 6 characters in length
- No database schema changes required
2025-12-06 10:01:05 -05:00
DJP
4724fa3333 feat: Add A2-A3 batch upload completion notification template. 2025-12-03 17:05:39 -05:00
DJP
bce739d116 feat: Automatically set asset validity dates in the metadata extractor and update the agency name in field mappings. 2025-12-03 14:28:59 -05:00
DJP
85b39c49cb Add debug script for B1 pagination issue 2025-12-02 09:44:10 -05:00
DJP
ea2e313185 feat: Add systemd services and timers for daily and weekly database backups, introduce a token retrieval script, and update the backup guide documentation. 2025-12-01 22:19:24 -05:00
DJP
9b24ca1594 Add systemd service files for Prod Orchestrator and CreativeX Service 2025-12-01 16:09:26 -05:00
DJP
ca11c0dbb4 Fix NameError: Restore missing import in dam_client.py 2025-12-01 15:31:30 -05:00
DJP
69ffdeb14e Add CreativeX service runner and fix dam_client indentation 2025-12-01 15:29:27 -05:00
DJP
5f389f6490 Implement robust Box name sanitization in shared/common.py 2025-12-01 15:23:01 -05:00
DJP
499399d6c6 Sanitize subfolder names in dam_client recursive search 2025-12-01 15:16:55 -05:00
DJP
2a8129f428 Fix Box folder naming: sanitize forward slashes in campaign names 2025-12-01 15:13:58 -05:00
DJP
4d1e71d978 Implement true recursion in DAM client: treat items without extensions as folders 2025-12-01 14:54:21 -05:00
DJP
67c8765512 Add fallback check to B1->B2 script: skip assets with no file extension 2025-12-01 14:22:59 -05:00
DJP
0f6e816255 Fix recursive download in B1->B2 script: improve folder identification and add safeguards 2025-12-01 14:17:14 -05:00
DJP
4c691dda30 Update CSV format to combine Campaign Number and Name, add manual regeneration script 2025-12-01 09:22:32 -05:00
DJP
db0dceb19c Add --auth-pfx-v2 and --env options to update_campaign_status.py 2025-11-27 06:58:39 -05:00
DJP
4259b1073c Add --auth-pfx-v2 support to A4 Box Uploader 2025-11-26 17:01:34 -05:00
DJP
17648d9051 Fix test_connection to not use non-existent /v6 endpoint 2025-11-26 16:58:55 -05:00
DJP
532699f634 Show full OAuth tokens in logs for debugging 2025-11-26 16:44:07 -05:00
DJP
cda33e8c8d Switch production orchestrator back to mTLS V2 (--auth-pfx-v2) 2025-11-26 16:32:43 -05:00
DJP
b2c8e01d4b Add token preview logging for OAuth tokens 2025-11-26 16:32:10 -05:00
DJP
c7ddf1d45f Add verbose logging for all DAM API requests 2025-11-26 16:26:15 -05:00
DJP
d68e149d1e Switch production orchestrator to use legacy mTLS (--auth-pfx) 2025-11-26 16:05:10 -05:00
DJP
37cab9d902 Add support for loading environment-specific .env files (prod/dev) 2025-11-26 15:44:09 -05:00
DJP
c1bd575bf9 Add --auth-pfx-v2 support to A4 Webhook Monitor for production 2025-11-26 15:39:00 -05:00
DJP
c1df6a7712 Fix A4 Webhook Monitor to use --auth-pfx for production 2025-11-26 15:38:06 -05:00
DJP
87b593c5f9 Fix syntax errors in orchestrator-prod.py args 2025-11-26 15:34:14 -05:00
DJP
c53e79cbaf Add production orchestrator configuration 2025-11-26 15:31:32 -05:00
DJP
07bce09d65 Fix B1→B2 bug: total_assets referenced before assignment 2025-11-26 15:07:27 -05:00
DJP
8ca44fcf1e Add metadata diagnostic tool for troubleshooting field issues 2025-11-26 14:52:39 -05:00
DJP
f9c11ef3f5 Fix misleading log message for A5 campaigns with no rejections 2025-11-26 14:45:48 -05:00
DJP
795e4e7d96 Improve A5 notification logic to handle status changes 2025-11-26 14:26:36 -05:00
DJP
0f1c3dd0ec Prevent duplicate 'no rejections' emails for A5 campaigns 2025-11-26 14:25:57 -05:00
DJP
936071d7ad Change script intervals from 5 to 3 minutes for faster processing 2025-11-26 14:18:45 -05:00
DJP
f15ae9a8d1 Stream full script output to console in real-time 2025-11-26 14:04:15 -05:00
DJP
d6b68af5d5 Fix A5→A6 to also use OAuth authentication 2025-11-26 14:03:07 -05:00
DJP
16527f6e43 Temporarily disable mTLS auth, use OAuth for troubleshooting 2025-11-26 13:59:48 -05:00