Compare commits
12 commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
de04cfc8fb | ||
|
|
37097d2148 | ||
|
|
2ec22c62a5 | ||
|
|
7412019053 | ||
|
|
7deb9db0a5 | ||
|
|
ad714d4b14 | ||
|
|
e327502723 | ||
|
|
a2f1954038 | ||
|
|
b89a44984d | ||
|
|
1b4e1a1cbc | ||
|
|
26363f772d | ||
|
|
444ac7ac6d |
17 changed files with 2872 additions and 163 deletions
|
|
@ -126,16 +126,22 @@ tail -f logs/cron_a1_a2.log
|
|||
- Rejection comment extraction (A5→A6 specific)
|
||||
|
||||
- **metadata_extractor_mvp.py**: Field mapping and metadata transformation
|
||||
- Loads 27 MVP fields from `config/field_mappings.yaml`
|
||||
- Handles filename-based updates
|
||||
- Loads MVP fields from environment-specific config (`field_mappings_ppr.yaml` or `field_mappings_prod.yaml`)
|
||||
- **Two tracking modes:** Full inheritance (from master metadata) and folder-only (`-N` suffix, uses `config/asset_representation_template.json` as base)
|
||||
- Handles filename-based updates, forced values, defaults, and asset type overrides
|
||||
- Asset type overrides (e.g., EOL) can set/remove fields with final precedence
|
||||
- Force-sets required values (e.g., STATE = "Local")
|
||||
- Uses DomainValue format for domained fields when setting values on template fields
|
||||
|
||||
### Configuration Architecture
|
||||
|
||||
**Hierarchical config system:**
|
||||
- `.env`: Environment variables (credentials, never committed)
|
||||
- `config/config.yaml`: Main configuration (references .env vars)
|
||||
- `config/field_mappings.yaml`: Editable field definitions (add/remove fields without code changes)
|
||||
- `config/field_mappings_ppr.yaml`: PPR field definitions (auto-loaded when DAM URL contains 'ppr')
|
||||
- `config/field_mappings_prod.yaml`: PROD field definitions (auto-loaded otherwise)
|
||||
- `config/asset_type_mappings.yaml`: 3-letter code to DAM code mappings (e.g., EOL -> externallegalopinion)
|
||||
- `config/asset_representation_template.json`: Reference template for folder-only mode (-N flag), contains full field metadata structure
|
||||
- `../Box-config.json`: Box JWT credentials (one directory up from Python-Version)
|
||||
|
||||
**Important**: Box-config.json MUST be located at `../Box-config.json` (one folder up). This is hardcoded in config.yaml as `rsa_private_key_path: ../Box-config.json`.
|
||||
|
|
@ -181,8 +187,11 @@ tail -f logs/cron_a1_a2.log
|
|||
**A2→A3 (Upload from Box):**
|
||||
- Polls `BOX_ROOT_FOLDER_A2_A3` (348526703108) for new files
|
||||
- Parses tracking ID from filename (V2 format)
|
||||
- Loads master metadata from database
|
||||
- Two tracking modes:
|
||||
- **Full inheritance**: Loads master metadata from database, inherits all fields
|
||||
- **Folder-only** (`-N` suffix): Uses `config/asset_representation_template.json` as base, populates from filename
|
||||
- Updates Description, Language, State fields from filename
|
||||
- Applies asset type overrides (e.g., EOL sets Agency="-", Languages="Global", IPRights="Yes", removes validity dates)
|
||||
- Deletes file from Box after successful upload
|
||||
- Updates campaign status A2→A3 when ALL assets uploaded
|
||||
|
||||
|
|
@ -219,7 +228,10 @@ New workflow scripts should follow this pattern:
|
|||
|
||||
### Modifying Field Mappings
|
||||
|
||||
**To add/remove fields, edit `config/field_mappings.yaml`:**
|
||||
**To add/remove fields, edit the environment-specific file:**
|
||||
- PPR: `config/field_mappings_ppr.yaml`
|
||||
- PROD: `config/field_mappings_prod.yaml`
|
||||
|
||||
```yaml
|
||||
mvp_fields:
|
||||
- FERRERO.FIELD.NEW_FIELD_NAME # Add here
|
||||
|
|
@ -228,6 +240,19 @@ mvp_fields:
|
|||
|
||||
**No code changes required** - the system dynamically loads fields at runtime.
|
||||
|
||||
### Asset Type Overrides
|
||||
|
||||
To add field overrides for a specific asset type, add an `asset_type_overrides` section to the field mappings file:
|
||||
```yaml
|
||||
asset_type_overrides:
|
||||
EOL: # Keyed by 3-letter asset type code
|
||||
FERRERO.MARKETING.FIELD.AGENCY NAME: "-"
|
||||
MAIN_LANGUAGES: "Global"
|
||||
FERRERO.FIELD.ASSET VALIDITY START PERIOD: "" # Empty string removes the field
|
||||
```
|
||||
|
||||
Overrides run after all other field processing (forced values, defaults) and take final precedence. An empty string value removes the field entirely from the payload.
|
||||
|
||||
### Database Queries
|
||||
|
||||
Common patterns used in the codebase:
|
||||
|
|
@ -301,7 +326,10 @@ except Exception as e:
|
|||
├── .env # Environment variables
|
||||
├── config/
|
||||
│ ├── config.yaml
|
||||
│ ├── field_mappings.yaml
|
||||
│ ├── field_mappings_ppr.yaml
|
||||
│ ├── field_mappings_prod.yaml
|
||||
│ ├── asset_type_mappings.yaml
|
||||
│ ├── asset_representation_template.json
|
||||
│ └── certificates/
|
||||
│ └── dam-mtls-dev.pfx
|
||||
├── database/
|
||||
|
|
|
|||
378
Python-Version/OPTION1_MULTIPLE_TRACKING_IDS.md
Normal file
378
Python-Version/OPTION1_MULTIPLE_TRACKING_IDS.md
Normal file
|
|
@ -0,0 +1,378 @@
|
|||
# Option 1: Multiple Tracking IDs in Filename - Implementation Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Allow a single derivative/localized asset to reference multiple master assets by including multiple tracking IDs in the filename.
|
||||
|
||||
**Example Filename:**
|
||||
```
|
||||
1234568_ROC_STRANGER-THINGS_SND_6S_16x9_REF_DE_de_BqB8vo+SfUQ7m+laRJo0.jpg
|
||||
^^^^^^^^^^^^^^^^^
|
||||
Multiple tracking IDs
|
||||
```
|
||||
|
||||
**Delimiter:** Use `+` to separate multiple tracking IDs (could also use `,` or `_`)
|
||||
|
||||
---
|
||||
|
||||
## Changes Required
|
||||
|
||||
### 1️⃣ Filename Parser (`scripts/shared/filename_parser.py`)
|
||||
|
||||
**Current Code (line ~182):**
|
||||
```python
|
||||
# Tracking ID: 6 alphanumeric, optionally with -N suffix
|
||||
elif re.match(r'^[a-zA-Z0-9]{6}(-N)?$', part):
|
||||
tracking = part
|
||||
tracking_mode = 'full'
|
||||
base_tracking_id = tracking
|
||||
|
||||
if tracking.endswith('-N'):
|
||||
tracking_mode = 'folder_only'
|
||||
base_tracking_id = tracking[:-2] # Strip -N suffix
|
||||
|
||||
parsed['tracking_id'] = base_tracking_id
|
||||
parsed['tracking_mode'] = tracking_mode
|
||||
parsed['tracking_id_with_suffix'] = tracking
|
||||
logger.debug("Found tracking ID: {}".format(tracking))
|
||||
index += 1
|
||||
```
|
||||
|
||||
**Modified Code:**
|
||||
```python
|
||||
# Tracking ID(s): 6 alphanumeric, optionally with -N suffix
|
||||
# Supports multiple IDs separated by + (e.g., "BqB8vo+SfUQ7m+laRJo0")
|
||||
elif re.match(r'^[a-zA-Z0-9]{6}(-N)?(\+[a-zA-Z0-9]{6}(-N)?)*$', part):
|
||||
tracking_ids = []
|
||||
tracking_modes = []
|
||||
tracking_ids_with_suffix = []
|
||||
|
||||
# Split by + delimiter to get all tracking IDs
|
||||
id_parts = part.split('+')
|
||||
|
||||
for tracking in id_parts:
|
||||
tracking_mode = 'full'
|
||||
base_tracking_id = tracking
|
||||
|
||||
if tracking.endswith('-N'):
|
||||
tracking_mode = 'folder_only'
|
||||
base_tracking_id = tracking[:-2] # Strip -N suffix
|
||||
logger.info("Detected folder-only tracking ID: {} (base: {})".format(tracking, base_tracking_id))
|
||||
|
||||
tracking_ids.append(base_tracking_id)
|
||||
tracking_modes.append(tracking_mode)
|
||||
tracking_ids_with_suffix.append(tracking)
|
||||
|
||||
# Store primary (first) tracking ID for backward compatibility
|
||||
parsed['tracking_id'] = tracking_ids[0]
|
||||
parsed['tracking_mode'] = tracking_modes[0]
|
||||
parsed['tracking_id_with_suffix'] = tracking_ids_with_suffix[0]
|
||||
|
||||
# Store all tracking IDs for multi-master support
|
||||
parsed['tracking_ids'] = tracking_ids
|
||||
parsed['tracking_modes'] = tracking_modes
|
||||
parsed['tracking_ids_with_suffix'] = tracking_ids_with_suffix
|
||||
parsed['has_multiple_masters'] = len(tracking_ids) > 1
|
||||
|
||||
logger.debug("Found {} tracking ID(s): {}".format(len(tracking_ids), ', '.join(tracking_ids)))
|
||||
index += 1
|
||||
```
|
||||
|
||||
**Key Changes:**
|
||||
- Updated regex to match multiple IDs: `^[a-zA-Z0-9]{6}(-N)?(\+[a-zA-Z0-9]{6}(-N)?)*$`
|
||||
- Split on `+` delimiter
|
||||
- Store primary ID for backward compatibility
|
||||
- Add new fields: `tracking_ids`, `has_multiple_masters`
|
||||
|
||||
---
|
||||
|
||||
### 2️⃣ A2→A3 Upload Script (`scripts/a2_to_a3_upload_polling.py`)
|
||||
|
||||
**Current Code (line ~97):**
|
||||
```python
|
||||
# 2. Load master metadata from database
|
||||
master_asset = db.get_master_asset(tracking_id)
|
||||
|
||||
if not master_asset:
|
||||
raise ValueError("No master asset for tracking ID: {}".format(tracking_id))
|
||||
```
|
||||
|
||||
**Modified Code:**
|
||||
```python
|
||||
# 2. Load master metadata from database (support multiple tracking IDs)
|
||||
tracking_ids = parsed.get('tracking_ids', [tracking_id]) # Get all tracking IDs or fallback to single
|
||||
has_multiple_masters = parsed.get('has_multiple_masters', False)
|
||||
|
||||
# Load all master assets
|
||||
master_assets = []
|
||||
master_opentext_ids = []
|
||||
|
||||
if has_multiple_masters:
|
||||
logger.info("Multiple master assets detected: {}".format(', '.join(tracking_ids)))
|
||||
|
||||
for tid in tracking_ids:
|
||||
master = db.get_master_asset(tid)
|
||||
if not master:
|
||||
logger.warning("Master asset not found for tracking ID: {}".format(tid))
|
||||
continue
|
||||
master_assets.append(master)
|
||||
master_opentext_ids.append(master['opentext_id'])
|
||||
|
||||
if not master_assets:
|
||||
raise ValueError("No master assets found for tracking IDs: {}".format(', '.join(tracking_ids)))
|
||||
|
||||
# Use first master for metadata inheritance (could enhance this later)
|
||||
master_asset = master_assets[0]
|
||||
logger.info("Using primary master {} for metadata, linking all {} masters".format(
|
||||
tracking_ids[0], len(master_assets)))
|
||||
else:
|
||||
# Single master (backward compatible)
|
||||
master_asset = db.get_master_asset(tracking_id)
|
||||
if not master_asset:
|
||||
raise ValueError("No master asset for tracking ID: {}".format(tracking_id))
|
||||
master_opentext_ids = [master_asset['opentext_id']]
|
||||
```
|
||||
|
||||
**Current Code (line ~194):**
|
||||
```python
|
||||
asset_rep = mvp_extractor.build_mvp_asset_representation(
|
||||
master_metadata=master_asset['full_metadata'],
|
||||
clean_filename=clean_filename,
|
||||
parsed_filename=parsed,
|
||||
box_metadata=box_metadata,
|
||||
tracking_mode=tracking_mode,
|
||||
master_opentext_id=master_asset['opentext_id'] # Single ID
|
||||
)
|
||||
```
|
||||
|
||||
**Modified Code:**
|
||||
```python
|
||||
# Pass all master opentext IDs (support multiple)
|
||||
asset_rep = mvp_extractor.build_mvp_asset_representation(
|
||||
master_metadata=master_asset['full_metadata'],
|
||||
clean_filename=clean_filename,
|
||||
parsed_filename=parsed,
|
||||
box_metadata=box_metadata,
|
||||
tracking_mode=tracking_mode,
|
||||
master_opentext_id=master_asset['opentext_id'], # Primary for ARTESIA.FIELD.ASSET_ID
|
||||
master_opentext_ids=master_opentext_ids # All IDs for MASTERASSETIDS field
|
||||
)
|
||||
```
|
||||
|
||||
**Key Changes:**
|
||||
- Extract multiple tracking IDs from parsed data
|
||||
- Look up all master assets in database
|
||||
- Collect all master opentext_ids
|
||||
- Pass list to metadata extractor
|
||||
|
||||
---
|
||||
|
||||
### 3️⃣ Metadata Extractor (`scripts/shared/metadata_extractor_mvp.py`)
|
||||
|
||||
**Current Method Signature (line ~97):**
|
||||
```python
|
||||
def build_mvp_asset_representation(self, master_metadata, clean_filename,
|
||||
parsed_filename, box_metadata=None,
|
||||
tracking_mode='full', master_opentext_id=None):
|
||||
```
|
||||
|
||||
**Modified Method Signature:**
|
||||
```python
|
||||
def build_mvp_asset_representation(self, master_metadata, clean_filename,
|
||||
parsed_filename, box_metadata=None,
|
||||
tracking_mode='full', master_opentext_id=None,
|
||||
master_opentext_ids=None):
|
||||
```
|
||||
|
||||
**Current Code (line ~139):**
|
||||
```python
|
||||
if master_opentext_id:
|
||||
mvp_fields = self._add_master_asset_id_field(mvp_fields, master_opentext_id)
|
||||
logger.info("Added Master Asset ID field: {}".format(master_opentext_id))
|
||||
```
|
||||
|
||||
**Modified Code:**
|
||||
```python
|
||||
# Add Master Asset ID field(s) if provided (derivative tracking)
|
||||
if master_opentext_id:
|
||||
mvp_fields = self._add_master_asset_id_field(mvp_fields, master_opentext_id)
|
||||
logger.info("Added Master Asset ID field: {}".format(master_opentext_id))
|
||||
|
||||
# Add MASTERASSETIDS tabular field with all master IDs (support multiple)
|
||||
if master_opentext_ids and len(master_opentext_ids) > 0:
|
||||
mvp_fields = self._add_master_asset_ids_field(mvp_fields, master_opentext_ids)
|
||||
logger.info("Added MASTERASSETIDS field with {} value(s)".format(len(master_opentext_ids)))
|
||||
```
|
||||
|
||||
**New Method (add after `_add_master_asset_id_field`):**
|
||||
```python
|
||||
def _add_master_asset_ids_field(self, mvp_fields, master_opentext_ids):
|
||||
"""
|
||||
Add FERRERO.MASTERASSETIDS tabular field with multiple master asset IDs
|
||||
Supports Many-to-Many relationship between derivatives and masters
|
||||
|
||||
Args:
|
||||
mvp_fields: List of MVP fields
|
||||
master_opentext_ids: List of DAM Asset IDs of master assets
|
||||
|
||||
Returns:
|
||||
Updated mvp_fields list with FERRERO.MASTERASSETIDS
|
||||
"""
|
||||
if not master_opentext_ids or len(master_opentext_ids) == 0:
|
||||
logger.info("No master_opentext_ids provided - skipping FERRERO.MASTERASSETIDS field")
|
||||
return mvp_fields
|
||||
|
||||
# Check if field already exists
|
||||
for field in mvp_fields:
|
||||
if self._get_field_id(field) == 'FERRERO.MASTERASSETIDS':
|
||||
logger.info("FERRERO.MASTERASSETIDS already present")
|
||||
return mvp_fields
|
||||
|
||||
# Build values array with all master asset IDs
|
||||
values = []
|
||||
for master_id in master_opentext_ids:
|
||||
values.append({
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'is_locked': False,
|
||||
'value': {
|
||||
'field_value': {
|
||||
'type': 'string',
|
||||
'value': master_id
|
||||
},
|
||||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
})
|
||||
|
||||
# Create tabular field
|
||||
new_field = {
|
||||
'id': 'FERRERO.MASTERASSETIDS',
|
||||
'parent_table_id': 'FERRERO.TABULAR.FIELD.MASTERASSETIDS',
|
||||
'type': 'com.artesia.metadata.MetadataTableField',
|
||||
'values': values
|
||||
}
|
||||
|
||||
mvp_fields.append(new_field)
|
||||
logger.info("Added FERRERO.MASTERASSETIDS field with {} master asset ID(s): {}".format(
|
||||
len(values), ', '.join(master_opentext_ids[:3]) + ('...' if len(master_opentext_ids) > 3 else '')))
|
||||
|
||||
return mvp_fields
|
||||
```
|
||||
|
||||
**Key Changes:**
|
||||
- Add `master_opentext_ids` parameter (list)
|
||||
- New method `_add_master_asset_ids_field` that accepts a list
|
||||
- Builds `values` array with all master IDs
|
||||
- Backward compatible (still uses single `master_opentext_id` for ARTESIA.FIELD.ASSET_ID)
|
||||
|
||||
---
|
||||
|
||||
## Testing Examples
|
||||
|
||||
### Single Master (Backward Compatible)
|
||||
**Filename:** `1234568_ROC_STRANGER-THINGS_SND_6S_16x9_REF_DE_de_BqB8vo.jpg`
|
||||
|
||||
**Parsed:**
|
||||
```python
|
||||
{
|
||||
'tracking_id': 'BqB8vo',
|
||||
'tracking_ids': ['BqB8vo'],
|
||||
'has_multiple_masters': False
|
||||
}
|
||||
```
|
||||
|
||||
**Result:** Single ID in MASTERASSETIDS field (current behavior)
|
||||
|
||||
---
|
||||
|
||||
### Multiple Masters (New Feature)
|
||||
**Filename:** `1234568_ROC_STRANGER-THINGS_SND_6S_16x9_REF_DE_de_BqB8vo+SfUQ7m+laRJo0.jpg`
|
||||
|
||||
**Parsed:**
|
||||
```python
|
||||
{
|
||||
'tracking_id': 'BqB8vo', # Primary (for backward compatibility)
|
||||
'tracking_ids': ['BqB8vo', 'SfUQ7m', 'laRJo0'],
|
||||
'has_multiple_masters': True
|
||||
}
|
||||
```
|
||||
|
||||
**Database Lookups:**
|
||||
- BqB8vo → fc5c389776516bb58044c7d4bf479da458599baf
|
||||
- SfUQ7m → ad3948d72ea8550a338a600ae87a1bdd1968b066
|
||||
- laRJo0 → 020d76f957ec9f4ec0b18035a2d012cd3fd376c2
|
||||
|
||||
**Result:** 3 IDs in MASTERASSETIDS field values array
|
||||
|
||||
---
|
||||
|
||||
## Migration Path
|
||||
|
||||
1. **Phase 1 - Implement Code** (No Breaking Changes)
|
||||
- Add changes to all 3 files
|
||||
- Test with single tracking ID (should work exactly as before)
|
||||
- Backward compatible with existing filenames
|
||||
|
||||
2. **Phase 2 - Test Multiple IDs**
|
||||
- Create test file with multiple tracking IDs
|
||||
- Upload to PPR with `--dryrun`
|
||||
- Verify 3 values in MASTERASSETIDS field
|
||||
|
||||
3. **Phase 3 - Agency Tool Integration**
|
||||
- Agency tool generates filenames with `+` delimiter
|
||||
- Agency tool uses multiple tracking IDs when needed
|
||||
- Most files will still have single tracking ID (normal case)
|
||||
|
||||
4. **Phase 4 - Production Deployment**
|
||||
- Enable in PROD after testing in PPR
|
||||
- Update field in PROD DAM schema first
|
||||
- Deploy code changes
|
||||
|
||||
---
|
||||
|
||||
## Alternative Delimiters
|
||||
|
||||
If `+` causes issues, alternatives:
|
||||
|
||||
| Delimiter | Example | Notes |
|
||||
|-----------|---------|-------|
|
||||
| `+` | `BqB8vo+SfUQ7m` | ✅ Recommended (clear separator) |
|
||||
| `,` | `BqB8vo,SfUQ7m` | ⚠️ Might conflict with CSV exports |
|
||||
| `_` | `BqB8vo_SfUQ7m` | ⚠️ Already used in filename structure |
|
||||
| `~` | `BqB8vo~SfUQ7m` | ✅ Alternative if + causes issues |
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
**What happens if one tracking ID is not found?**
|
||||
|
||||
```python
|
||||
# Option A: Skip missing masters (log warning)
|
||||
for tid in tracking_ids:
|
||||
master = db.get_master_asset(tid)
|
||||
if not master:
|
||||
logger.warning("Master asset not found for tracking ID: {}".format(tid))
|
||||
continue # Skip this one, continue with others
|
||||
|
||||
# Option B: Fail entire upload (strict)
|
||||
for tid in tracking_ids:
|
||||
master = db.get_master_asset(tid)
|
||||
if not master:
|
||||
raise ValueError("Master asset not found for tracking ID: {}".format(tid))
|
||||
```
|
||||
|
||||
**Recommendation:** Use Option A (skip missing) - derivative still uploads with available master links.
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
**Files to Modify:**
|
||||
1. `scripts/shared/filename_parser.py` - Parse multiple tracking IDs
|
||||
2. `scripts/a2_to_a3_upload_polling.py` - Look up multiple masters
|
||||
3. `scripts/shared/metadata_extractor_mvp.py` - Add all IDs to field
|
||||
|
||||
**Backward Compatible:** ✅ Yes - existing single-ID filenames work exactly as before
|
||||
|
||||
**Ready to Implement:** This document provides all code changes needed.
|
||||
179
Python-Version/PPR_MULTIPLE_TRACKING_IDS_IMPLEMENTED.md
Normal file
179
Python-Version/PPR_MULTIPLE_TRACKING_IDS_IMPLEMENTED.md
Normal file
|
|
@ -0,0 +1,179 @@
|
|||
# PPR-Only Multiple Tracking IDs - Implementation Complete
|
||||
|
||||
## ✅ Changes Implemented
|
||||
|
||||
Multiple tracking IDs feature is now **ACTIVE in PPR** and **DISABLED in PROD** via environment detection.
|
||||
|
||||
---
|
||||
|
||||
## Files Modified
|
||||
|
||||
### 1. `scripts/shared/filename_parser.py`
|
||||
- Added `__init__` method with DAM URL parameter
|
||||
- Added `_is_ppr_environment()` method
|
||||
- Updated tracking ID parsing to:
|
||||
- **PPR**: Parse multiple IDs separated by `+` (e.g., `BqB8vo+SfUQ7m+laRJo0`)
|
||||
- **PROD**: Use only first ID (backward compatible)
|
||||
|
||||
### 2. `scripts/a2_to_a3_upload_polling.py`
|
||||
- Pass DAM URL to FilenameParser for environment detection
|
||||
- Loop through all tracking IDs (PPR) or single ID (PROD)
|
||||
- Look up all master assets in database
|
||||
- Collect all `opentext_id` values
|
||||
- Pass list to metadata extractor
|
||||
|
||||
### 3. `scripts/shared/metadata_extractor_mvp.py`
|
||||
- Added `master_opentext_ids` parameter (list)
|
||||
- New method: `_add_master_asset_ids_field()` to handle multiple IDs
|
||||
- Builds `values` array with all master IDs
|
||||
|
||||
---
|
||||
|
||||
## Environment Detection Logic
|
||||
|
||||
**PPR Environment:**
|
||||
- DAM URL contains: `ppr.dam.ferrero.com`
|
||||
- Multiple tracking IDs: ✅ **ENABLED**
|
||||
- Filename format: `1234568_ROC_ST_SND_6S_16x9_REF_DE_de_BqB8vo+SfUQ7m.jpg`
|
||||
|
||||
**PROD Environment:**
|
||||
- DAM URL contains: `dam.ferrero.com` (not ppr)
|
||||
- Multiple tracking IDs: ❌ **DISABLED**
|
||||
- Filename format: `1234568_ROC_ST_SND_6S_16x9_REF_DE_de_BqB8vo.jpg` (single ID)
|
||||
- If multiple IDs provided, uses FIRST ID only with warning
|
||||
|
||||
---
|
||||
|
||||
## Testing in PPR
|
||||
|
||||
### Test 1: Single Tracking ID (Backward Compatible)
|
||||
|
||||
**Filename:**
|
||||
```
|
||||
1234568_ROC_STRANGER-THINGS_SND_6S_16x9_REF_DE_de_BqB8vo.jpg
|
||||
```
|
||||
|
||||
**Expected Result:**
|
||||
- Parses as single tracking ID
|
||||
- One master asset looked up
|
||||
- One value in MASTERASSETIDS field
|
||||
- ✅ Works exactly as before
|
||||
|
||||
### Test 2: Multiple Tracking IDs (New Feature)
|
||||
|
||||
**Filename:**
|
||||
```
|
||||
1234568_ROC_STRANGER-THINGS_SND_6S_16x9_REF_DE_de_BqB8vo+SfUQ7m+laRJo0.jpg
|
||||
```
|
||||
|
||||
**Expected Result:**
|
||||
- PPR environment detected
|
||||
- Parses 3 tracking IDs: `BqB8vo`, `SfUQ7m`, `laRJo0`
|
||||
- Looks up 3 master assets in database
|
||||
- Gets 3 opentext_ids:
|
||||
- `fc5c389776516bb58044c7d4bf479da458599baf`
|
||||
- `ad3948d72ea8550a338a600ae87a1bdd1968b066`
|
||||
- `020d76f957ec9f4ec0b18035a2d012cd3fd376c2`
|
||||
- Creates MASTERASSETIDS field with 3 values
|
||||
|
||||
**Log Output:**
|
||||
```
|
||||
PPR Environment - Multiple tracking IDs detected: 3
|
||||
Parsed 3 tracking IDs: BqB8vo, SfUQ7m, laRJo0
|
||||
PPR - Multiple master assets detected: BqB8vo, SfUQ7m, laRJo0
|
||||
Using primary master BqB8vo for metadata, linking 3 total masters
|
||||
PPR - Added MASTERASSETIDS field with 3 master IDs
|
||||
Added FERRERO.MASTERASSETIDS field with 3 master asset ID(s): fc5c389776516bb58044c7d4bf479da458599baf, ad3948d72ea8550a338a600ae87a1bdd1968b066, 020d76f957ec9f4ec0b18035a2d012cd3fd376c2
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Test Commands
|
||||
|
||||
### Dry Run (Recommended First)
|
||||
```bash
|
||||
python scripts/a2_to_a3_upload_polling.py --dryrun
|
||||
```
|
||||
|
||||
Check the JSON output for:
|
||||
```json
|
||||
{
|
||||
"id": "FERRERO.MASTERASSETIDS",
|
||||
"parent_table_id": "FERRERO.TABULAR.FIELD.MASTERASSETIDS",
|
||||
"type": "com.artesia.metadata.MetadataTableField",
|
||||
"values": [
|
||||
{"value": {"field_value": {"value": "fc5c389776516bb58044c7d4bf479da458599baf"}}},
|
||||
{"value": {"field_value": {"value": "ad3948d72ea8550a338a600ae87a1bdd1968b066"}}},
|
||||
{"value": {"field_value": {"value": "020d76f957ec9f4ec0b18035a2d012cd3fd376c2"}}}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Real Upload to PPR
|
||||
```bash
|
||||
python scripts/a2_to_a3_upload_polling.py
|
||||
```
|
||||
|
||||
Then verify in PPR DAM:
|
||||
1. Search for the uploaded asset
|
||||
2. Open metadata
|
||||
3. Check "Master Asset IDs" tabular field
|
||||
4. Should show multiple rows
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Missing Master Assets:**
|
||||
- If one tracking ID is not found in database, it's skipped with warning
|
||||
- Derivative still uploads with available master links
|
||||
- Log message: `Master asset not found for tracking ID: xyz - skipping`
|
||||
|
||||
**PROD Environment with Multiple IDs:**
|
||||
- Uses FIRST tracking ID only
|
||||
- Logs warning: `PROD Environment - Multiple tracking IDs not supported, using first ID only`
|
||||
- Works as backward compatible (no errors)
|
||||
|
||||
---
|
||||
|
||||
## Current Environment Check
|
||||
|
||||
Your `.env` file shows:
|
||||
```
|
||||
DAM_BASE_URL=https://ppr.dam.ferrero.com/otmmapi
|
||||
```
|
||||
|
||||
✅ **PPR Environment** - Multiple tracking IDs are **ENABLED**
|
||||
|
||||
---
|
||||
|
||||
## Agency Tool Requirements
|
||||
|
||||
To use multiple tracking IDs, the Agency tool needs to:
|
||||
1. Concatenate tracking IDs with `+` delimiter
|
||||
2. Example: `tracking_id_1 + "+" + tracking_id_2 + "+" + tracking_id_3`
|
||||
3. Place in filename: `{job}_{brand}_{...}_{tracking_ids}.{ext}`
|
||||
|
||||
**Most derivatives will still use single tracking ID** - this is only for special cases where one derivative references multiple masters.
|
||||
|
||||
---
|
||||
|
||||
## Production Safety
|
||||
|
||||
✅ **PROD is Protected:**
|
||||
- Environment detection prevents multiple IDs in PROD
|
||||
- If multiple IDs accidentally used, only first ID is processed
|
||||
- No breaking changes to PROD behavior
|
||||
- Fully backward compatible
|
||||
|
||||
---
|
||||
|
||||
## Ready to Test! 🚀
|
||||
|
||||
Your PPR environment is now ready to test multiple tracking IDs.
|
||||
|
||||
1. Create test file with multiple IDs
|
||||
2. Upload to Box: `DAM-UPLOAD/1234568/`
|
||||
3. Run with `--dryrun` first
|
||||
4. Verify JSON shows multiple values
|
||||
5. Real upload and check in PPR DAM
|
||||
|
|
@ -2,8 +2,8 @@
|
|||
|
||||
**Complete automated workflow for Ferrero DAM Content Scaling**
|
||||
|
||||
**Version:** 2.0
|
||||
**Last Updated:** November 5, 2025
|
||||
**Version:** 2.1
|
||||
**Last Updated:** March 31, 2026
|
||||
**Status:** ✅ Production Ready & Fully Tested
|
||||
|
||||
---
|
||||
|
|
@ -334,6 +334,12 @@ crontab -e
|
|||
7. Delete file from Box
|
||||
8. **Update A2→A3 when ALL campaign assets uploaded**
|
||||
|
||||
**Two tracking modes:**
|
||||
- **Full inheritance** (standard): Inherits all metadata from the master asset
|
||||
- **Folder-only** (`-N` suffix): Builds metadata from a reference template (`config/asset_representation_template.json`) and populates values from the filename. Used when the derivative only needs the upload folder from the master.
|
||||
|
||||
**Asset type overrides:** Certain asset types (e.g., EOL) trigger field overrides configured in the environment's field mappings file (e.g., Agency Name, Languages, IP Rights, validity dates).
|
||||
|
||||
**Box Folder:** 348526703108 (Agency Uploads)
|
||||
**Email:** a2_to_a3_file_uploaded, a2_to_a3_complete
|
||||
|
||||
|
|
@ -946,6 +952,35 @@ scp Box-config.json user@server:/opt/ferrero-automation/
|
|||
scp -r Python-Version/ user@server:/opt/ferrero-automation/
|
||||
```
|
||||
|
||||
### Field Mappings (Environment-Specific)
|
||||
|
||||
The system auto-detects the environment from the DAM URL and loads the appropriate config:
|
||||
|
||||
- **PPR:** `config/field_mappings_ppr.yaml` (pre-production, `ppr.dam.ferrero.com`)
|
||||
- **PROD:** `config/field_mappings_prod.yaml` (production, `dam.ferrero.com`)
|
||||
|
||||
Each file defines: MVP fields, filename update rules, forced values, defaults, and asset type overrides.
|
||||
|
||||
### Asset Type Mappings
|
||||
|
||||
`config/asset_type_mappings.yaml` maps 3-letter codes from the naming tool to DAM domain values (e.g., `EHI` -> `heroimage`, `EOL` -> `externallegalopinion`).
|
||||
|
||||
### Asset Representation Template
|
||||
|
||||
`config/asset_representation_template.json` is the reference template for folder-only mode (`-N` flag uploads). It contains the full field metadata structure that the DAM API requires for asset creation. This template was provided by the client and should be updated if the DAM metadata model changes.
|
||||
|
||||
### Asset Type Overrides (EOL Example)
|
||||
|
||||
Certain asset types trigger field overrides configured in the field mappings file. For example, **EOL (External Legal Opinion)** overrides:
|
||||
- Agency Name = "-"
|
||||
- Production House = "-"
|
||||
- Main Languages = "Global"
|
||||
- IP Rights = "Yes"
|
||||
- Licensing = "No"
|
||||
- Validity dates removed
|
||||
|
||||
These overrides are applied after all other field processing and take final precedence.
|
||||
|
||||
---
|
||||
|
||||
## 🗄️ Database
|
||||
|
|
|
|||
830
Python-Version/config/asset_representation_template.json
Normal file
830
Python-Version/config/asset_representation_template.json
Normal file
|
|
@ -0,0 +1,830 @@
|
|||
{
|
||||
"asset_resource": {
|
||||
"asset": {
|
||||
"metadata": {
|
||||
"metadata_element_list": [
|
||||
{
|
||||
"column_name": "ASSET_TYPE",
|
||||
"data_length": 30,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO.DOMAIN.MARKETING.ASSETTYPE",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": true,
|
||||
"id": "FERRERO.FIELD.MKTG.ASSET TYPE",
|
||||
"multilingual": false,
|
||||
"name": "Asset Type",
|
||||
"prompt": "Asset Type",
|
||||
"required": false,
|
||||
"restriction_id": 7242,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "FISCAL__YEAR",
|
||||
"data_length": 100,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO_DOMAIN_FISCAL_YEAR",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": true,
|
||||
"id": "FERRERO.FIELD.FISCAL YEAR",
|
||||
"multilingual": false,
|
||||
"name": "Release Fiscal Year",
|
||||
"prompt": "Release Fiscal Year",
|
||||
"required": true,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "ECOMMERCE",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "DESCR",
|
||||
"data_length": 249,
|
||||
"data_type": "CHAR",
|
||||
"description": "Descriptive information",
|
||||
"displayable": true,
|
||||
"domained": false,
|
||||
"edit_type": "TEXTAREA",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "ARTESIA.FIELD.ASSET DESCRIPTION",
|
||||
"multilingual": false,
|
||||
"name": "Description",
|
||||
"prompt": "Description",
|
||||
"required": false,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "UOIS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"value": {
|
||||
"type": "string",
|
||||
"value": ""
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "FLAVOUR",
|
||||
"data_length": 2000,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO.DOMAIN.MARKETING.FLAVOUR",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": true,
|
||||
"id": "FERRERO.FIELD.MARKETING.FLAVOUR",
|
||||
"multilingual": false,
|
||||
"name": "Flavour",
|
||||
"prompt": "Flavour",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "SIZE",
|
||||
"data_length": 2000,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO.DOMAIN.MARKETING.SIZE",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": true,
|
||||
"id": "FERRERO.FIELD.MARKETING.SIZE",
|
||||
"multilingual": false,
|
||||
"name": "Size",
|
||||
"prompt": "Size",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "ASSET_STATE",
|
||||
"data_length": 2000,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO.DOMAIN.GLOBAL.LOCAL",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO_NOTNULL",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": true,
|
||||
"id": "FERRERO.FIELD.STATE",
|
||||
"multilingual": false,
|
||||
"name": "Global/Local",
|
||||
"prompt": "Global/Local",
|
||||
"required": true,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "FERRERO_IC_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "NAME",
|
||||
"data_length": 256,
|
||||
"data_type": "CHAR",
|
||||
"description": "Original name of master object",
|
||||
"displayable": true,
|
||||
"domained": false,
|
||||
"edit_type": "SIMPLE",
|
||||
"editable": false,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "ARTESIA.FIELD.ASSET NAME",
|
||||
"multilingual": false,
|
||||
"name": "Asset Name",
|
||||
"prompt": "Asset Name",
|
||||
"required": false,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "UOIS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"value": {
|
||||
"type": "string",
|
||||
"value": ""
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "SUB_BRAND",
|
||||
"data_length": 2000,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO.DOMAIN.SUBBRAND",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": true,
|
||||
"id": "FERRERO.FIELD.SUB BRAND",
|
||||
"multilingual": false,
|
||||
"name": "Sub-Brands",
|
||||
"prompt": "Sub-Brands",
|
||||
"required": false,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "ECOMMERCE",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "VALIDATION_STARTING_DATE",
|
||||
"data_length": 20,
|
||||
"data_type": "DATE",
|
||||
"displayable": true,
|
||||
"domained": false,
|
||||
"edit_type": "DATE",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": true,
|
||||
"id": "FERRERO.FIELD.ASSET VALIDITY START PERIOD",
|
||||
"multilingual": false,
|
||||
"name": "Asset validity start period",
|
||||
"prompt": "Asset validity start period",
|
||||
"required": false,
|
||||
"restriction_id": 7242,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "FERRERO_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"value": {
|
||||
"type": "string",
|
||||
"value": ""
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "VALIDATION_ENDING_DATE",
|
||||
"data_length": 20,
|
||||
"data_type": "DATE",
|
||||
"displayable": true,
|
||||
"domained": false,
|
||||
"edit_type": "DATE",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": true,
|
||||
"id": "FERRERO.FIELD.ASSET VALIDITY END PERIOD",
|
||||
"multilingual": false,
|
||||
"name": "Asset validity end period",
|
||||
"prompt": "Asset validity end period",
|
||||
"required": false,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "FERRERO_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"value": {
|
||||
"type": "string",
|
||||
"value": ""
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "AGENCY_NAME",
|
||||
"data_length": 2000,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO.MARKETING.AGENCY_NAME",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.MARKETING.FIELD.AGENCY NAME",
|
||||
"multilingual": false,
|
||||
"name": "Agency Name",
|
||||
"prompt": "Agency Name",
|
||||
"required": false,
|
||||
"restriction_id": 7336,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "CREATIVE_LINK",
|
||||
"data_length": 2000,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domained": false,
|
||||
"edit_type": "TEXTAREA",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.FIELD.CREATIVEX LINK",
|
||||
"multilingual": false,
|
||||
"name": "CreativeX Hyperlink",
|
||||
"prompt": "CreativeX Hyperlink",
|
||||
"required": false,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "FERRERO_ASSET_CREATIVEX",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"value": {
|
||||
"type": "string",
|
||||
"value": ""
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "IP_RIGHTS",
|
||||
"data_length": 200,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO.MARKETING.IPRIGHTS",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": true,
|
||||
"id": "FERRERO.MARKET.FIELD.IPRIGHT",
|
||||
"multilingual": false,
|
||||
"name": "IP Rights",
|
||||
"prompt": "IP Rights",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "TOTAL_BUYOUT",
|
||||
"data_length": 200,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO.MARKETING.TOTAL_BUYOUT",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.MARKET.FIELD.BUYOUT",
|
||||
"multilingual": false,
|
||||
"name": "Touchpoint Scope",
|
||||
"prompt": "Touchpoint Scope",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "FERRERO_PROPERTY",
|
||||
"data_length": 2000,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO.MARKETING.FERRERO_PROPERTY",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.MARKET.FIELD.FERRERO PROPERTY",
|
||||
"multilingual": false,
|
||||
"name": "Ferrero Property",
|
||||
"prompt": "Ferrero Property",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "VID_AND_STAT_RIGHT",
|
||||
"data_length": 100,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO.MARKET.TECH_VALID",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.MARKET.VID_N_STAT",
|
||||
"multilingual": false,
|
||||
"name": "Video and Static Right",
|
||||
"prompt": "Video and Static Right",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "PRODUCTION_COMPANY",
|
||||
"data_length": 200,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO.DOMAIN.MARKETING.PRODUCT",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.MARKET.PROD_COMPANY",
|
||||
"multilingual": false,
|
||||
"name": "Production House",
|
||||
"prompt": "Production House",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "LICENSING",
|
||||
"data_length": 200,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO.MARKET.TECH_VALID",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.MARKET.FIELD.LICENSIN",
|
||||
"multilingual": false,
|
||||
"name": "Licensing",
|
||||
"prompt": "Licensing",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"cascading_group_id": "FERRERO.MARKET.CG.LICENSE",
|
||||
"column_name": "LICENSOR",
|
||||
"data_length": 2000,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domained": false,
|
||||
"edit_type": "CASCADING",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.MARKET.FIELD.LICENSE",
|
||||
"multilingual": false,
|
||||
"name": "License",
|
||||
"prompt": "License",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "SPOT_VERSION",
|
||||
"data_length": 100,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domain_id": "FERRERO.MARKETING.SPOT_VERSION",
|
||||
"domained": true,
|
||||
"edit_type": "COMBO",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.MARKETING.FIELD.SPOT_VERSION",
|
||||
"multilingual": false,
|
||||
"name": "Spot Version",
|
||||
"prompt": "Spot Version",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "DIRECTOR_NAME",
|
||||
"data_length": 100,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domained": false,
|
||||
"edit_type": "SIMPLE",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.MARKETING.FIELD.DIRECTOR_NAME",
|
||||
"multilingual": false,
|
||||
"name": "Director Name",
|
||||
"prompt": "Director Name",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "VIDEO_POST_PRODUCTION_COMPANY",
|
||||
"data_length": 200,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domained": false,
|
||||
"edit_type": "SIMPLE",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.MARKETING.FIELD.VIDEO_POST_PROD_COMPANY",
|
||||
"multilingual": false,
|
||||
"name": "Video Post-Production Company",
|
||||
"prompt": "Video Post-Production Company",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "VIDEO_COMPANY_DETAILS",
|
||||
"data_length": 2000,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domained": false,
|
||||
"edit_type": "TEXTAREA",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.MARKETING.FIELD.VID_POST_PROD_CONTACT",
|
||||
"multilingual": false,
|
||||
"name": "Video Post Production Company Contact Details",
|
||||
"prompt": "Video Post Production Company Contact Details",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "AUDIO_POST_PRODUCTION_COMPANY",
|
||||
"data_length": 200,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domained": false,
|
||||
"edit_type": "SIMPLE",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.MARKETING.FIELD.AUDIO_POST_PROD_COMPANY",
|
||||
"multilingual": false,
|
||||
"name": "Audio Post-Production Company",
|
||||
"prompt": "Audio Post-Production Company",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"column_name": "AUDIO_COMPANY_DETAILS",
|
||||
"data_length": 2000,
|
||||
"data_type": "CHAR",
|
||||
"displayable": true,
|
||||
"domained": false,
|
||||
"edit_type": "TEXTAREA",
|
||||
"editable": true,
|
||||
"enabled": true,
|
||||
"facetable": false,
|
||||
"id": "FERRERO.MARKETING.FIELD.AUDIO_POST_PROD_CONTACT",
|
||||
"multilingual": false,
|
||||
"name": "Audio Post Production Company Contact Details",
|
||||
"prompt": "Audio Post Production Company Contact Details",
|
||||
"required": false,
|
||||
"restriction_id": 5429,
|
||||
"scale": 0,
|
||||
"searchable": true,
|
||||
"searchable_scope_id": "1",
|
||||
"searchable_scope_num_id": 1,
|
||||
"sortable": true,
|
||||
"system_field": false,
|
||||
"table_name": "MARKETING_ASSET_DETAILS",
|
||||
"trigger_field": false,
|
||||
"type": "com.artesia.metadata.MetadataField",
|
||||
"value": {
|
||||
"cascading_domain_value": false,
|
||||
"domain_value": false,
|
||||
"is_locked": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "MAIN_LANGUAGES",
|
||||
"parent_table_id": "FERRERO.TABULAR.FIELD.MAIN LANGUAGES",
|
||||
"type": "com.artesia.metadata.MetadataTableField",
|
||||
"values": []
|
||||
},
|
||||
{
|
||||
"id": "FERRERO.FIELD.ASSETCOMPLIANCE",
|
||||
"parent_table_id": "FERRERO.TABULAR.FIELD.ASSETCOMPLIANCE",
|
||||
"type": "com.artesia.metadata.MetadataTableField",
|
||||
"values": []
|
||||
},
|
||||
{
|
||||
"id": "MARKETING_TAG",
|
||||
"parent_table_id": "FERRERO.TABULAR.FIELD.MARKETING_TAG",
|
||||
"type": "com.artesia.metadata.MetadataTableField",
|
||||
"values": []
|
||||
},
|
||||
{
|
||||
"id": "FERRERO.MARKET.FIELD.TYPE_VID",
|
||||
"parent_table_id": "FERRERO.TABULAR.VID_STAT_TYPE",
|
||||
"type": "com.artesia.metadata.MetadataTableField",
|
||||
"values": []
|
||||
}
|
||||
]
|
||||
},
|
||||
"metadata_model_id": "ECOMMERCE",
|
||||
"security_policy_list": [
|
||||
{
|
||||
"id": 1594
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -33,7 +33,6 @@ FPO: keyvisual # Front of Pack Image (alias for IMG)
|
|||
LGL: localguidelines # Local Guidelines
|
||||
LOG: ferrerologo # Logo
|
||||
MLF: marketingleaflet # Marketing Leaflet
|
||||
OLV: onlinevideodigitalvideo # On Line Video
|
||||
PAW: packartworks # Pack Artworks
|
||||
PKI: packshot # Pack Images (was packshot)
|
||||
POS: posm # POS Material
|
||||
|
|
@ -46,5 +45,8 @@ SGL: licenseshighlights # Styleguide Licenses
|
|||
TVC: tvc # TVC
|
||||
VIE: visualidentityelements # Visual Identity Elements
|
||||
|
||||
# External Legal Opinion
|
||||
EOL: externallegalopinion # External Legal Opinion (triggers field overrides)
|
||||
|
||||
# Note: If a 3-letter code is not in this mapping, it will be passed through as-is
|
||||
# and may fail DAM validation if the code doesn't exist in DAM's domain
|
||||
|
|
|
|||
|
|
@ -76,3 +76,16 @@ defaults:
|
|||
FERRERO.MARKETING.FIELD.VIDEO_POST_PROD_COMPANY: "Oliver Marketing Ltd"
|
||||
FERRERO.MARKETING.FIELD.AUDIO_POST_PROD_COMPANY: "Oliver Marketing Ltd"
|
||||
FERRERO.MARKET.PROD_COMPANY: "-" # Production House
|
||||
|
||||
# Asset type overrides (keyed by 3-letter asset type code)
|
||||
# Applied AFTER normal field updates and forced values
|
||||
# Overrides specific fields when a matching asset type is detected in the filename
|
||||
asset_type_overrides:
|
||||
EOL: # External Legal Opinion - selected as asset type in naming tool
|
||||
FERRERO.MARKETING.FIELD.AGENCY NAME: "-"
|
||||
FERRERO.MARKET.PROD_COMPANY: "-"
|
||||
MAIN_LANGUAGES: "Global"
|
||||
FERRERO.MARKET.FIELD.IPRIGHT: "Yes"
|
||||
FERRERO.MARKET.FIELD.LICENSIN: "No"
|
||||
FERRERO.FIELD.ASSET VALIDITY START PERIOD: "" # Remove validity dates for EOL
|
||||
FERRERO.FIELD.ASSET VALIDITY END PERIOD: "" # Remove validity dates for EOL
|
||||
|
|
|
|||
|
|
@ -4,8 +4,9 @@ import sys
|
|||
import psycopg2
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Load env vars
|
||||
load_dotenv('/Users/daveporter/Desktop/CODING-2024/Ferrero-Opentext/Python-Version/.env')
|
||||
# Load env vars from current directory
|
||||
script_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
load_dotenv(os.path.join(script_dir, '.env'))
|
||||
|
||||
try:
|
||||
conn = psycopg2.connect(
|
||||
|
|
|
|||
|
|
@ -216,16 +216,68 @@ def process_campaign(campaign, dam, box, db, notifier, config):
|
|||
else:
|
||||
logger.info("Processing: {}".format(asset_name))
|
||||
|
||||
# 1. Download from DAM
|
||||
# 1. Extract Global Campaign Reference (needed for tracking ID lookup)
|
||||
global_ref = db.extract_global_campaign_reference(asset, campaign_number)
|
||||
|
||||
# 2. Find existing tracking ID or generate new one
|
||||
# Handles re-processing: if campaign was reset to A1 after adding new masters,
|
||||
# existing assets keep their tracking IDs, new assets get new IDs
|
||||
tracking_result = db.find_or_create_tracking_id(
|
||||
opentext_id=asset_id,
|
||||
local_campaign_id=global_ref['local_campaign_id']
|
||||
)
|
||||
tracking_id = tracking_result['tracking_id']
|
||||
is_existing = tracking_result['is_existing']
|
||||
|
||||
if is_existing:
|
||||
# Asset already processed in a previous A1→A2 cycle
|
||||
existing_master = db.get_master_asset(tracking_id)
|
||||
if existing_master and existing_master.get('box_file_id'):
|
||||
logger.info("Re-processing: reusing tracking ID {} for existing asset {} (skipping download/upload)".format(
|
||||
tracking_id, asset_name))
|
||||
box_result = {
|
||||
'file_id': existing_master['box_file_id'],
|
||||
'url': existing_master['box_url']
|
||||
}
|
||||
|
||||
# Update database metadata (asset_data may have changed in DAM)
|
||||
db_result = db.store_master_asset(
|
||||
tracking_id=tracking_id,
|
||||
opentext_id=asset_id,
|
||||
asset_data=asset,
|
||||
box_file_id=box_result['file_id'],
|
||||
box_url=box_result['url'],
|
||||
upload_folder_id=final_folder_id,
|
||||
global_master_campaign_id=global_ref['global_master_campaign_id'],
|
||||
global_master_folder_id=global_ref['global_master_folder_id'],
|
||||
local_campaign_id=global_ref['local_campaign_id']
|
||||
)
|
||||
|
||||
if db_result['success']:
|
||||
processed_assets.append({
|
||||
'asset_id': asset_id,
|
||||
'asset_name': asset_name,
|
||||
'tracking_id': tracking_id,
|
||||
'box_file_id': box_result['file_id'],
|
||||
'box_url': box_result['url'],
|
||||
'is_existing': True
|
||||
})
|
||||
logger.info("✓ Existing asset confirmed: {} → {} (skipped)".format(asset_name, tracking_id))
|
||||
else:
|
||||
raise Exception("Database update failed for existing asset")
|
||||
|
||||
continue # Skip to next asset
|
||||
else:
|
||||
# Tracking ID exists but no usable Box info - process normally
|
||||
logger.info("Existing tracking ID {} found but no Box info - downloading/uploading".format(tracking_id))
|
||||
|
||||
# 3. Download from DAM (new assets or existing without Box info)
|
||||
file_path = dam.download_asset(
|
||||
asset_id,
|
||||
output_dir='temp/downloads/{}'.format(campaign_id)
|
||||
)
|
||||
|
||||
# 2. Generate tracking ID (regular files never start with 'M')
|
||||
tracking_id = db.generate_unique_tracking_id(is_master=False)
|
||||
|
||||
# 3. Upload to Box (preserve folder structure from DAM)
|
||||
# 4. Upload to Box (preserve folder structure from DAM)
|
||||
box_result = box.upload_with_tracking_id(
|
||||
file_path=file_path,
|
||||
campaign_id=campaign_number,
|
||||
|
|
@ -234,9 +286,6 @@ def process_campaign(campaign, dam, box, db, notifier, config):
|
|||
subfolder_path=folder_path
|
||||
)
|
||||
|
||||
# 4. Extract Global Campaign Reference and Local Campaign ID
|
||||
global_ref = db.extract_global_campaign_reference(asset, campaign_number)
|
||||
|
||||
# 5. Store in database
|
||||
db_result = db.store_master_asset(
|
||||
tracking_id=tracking_id,
|
||||
|
|
@ -275,9 +324,10 @@ def process_campaign(campaign, dam, box, db, notifier, config):
|
|||
'asset_name': asset_name,
|
||||
'tracking_id': tracking_id,
|
||||
'box_file_id': box_result['file_id'],
|
||||
'box_url': box_result['url']
|
||||
'box_url': box_result['url'],
|
||||
'is_existing': False
|
||||
})
|
||||
logger.info("✓ Success: {} → {}".format(asset_name, tracking_id))
|
||||
logger.info("✓ New asset: {} → {}".format(asset_name, tracking_id))
|
||||
else:
|
||||
raise Exception("Database storage failed")
|
||||
|
||||
|
|
@ -295,10 +345,17 @@ def process_campaign(campaign, dam, box, db, notifier, config):
|
|||
# CHECK: All assets processed successfully?
|
||||
all_done = len(processed_assets) == total_assets
|
||||
|
||||
# Count new vs existing assets
|
||||
new_assets = [a for a in processed_assets if not a.get('is_existing')]
|
||||
existing_assets = [a for a in processed_assets if a.get('is_existing')]
|
||||
|
||||
logger.info("")
|
||||
logger.info("Campaign {} Results:".format(campaign_id))
|
||||
logger.info(" Total: {}".format(total_assets))
|
||||
logger.info(" Successful: {}".format(len(processed_assets)))
|
||||
if existing_assets:
|
||||
logger.info(" - New assets: {}".format(len(new_assets)))
|
||||
logger.info(" - Existing assets (skipped download/upload): {}".format(len(existing_assets)))
|
||||
logger.info(" Failed: {}".format(len(failed_assets)))
|
||||
logger.info(" All Done: {}".format("YES" if all_done else "NO"))
|
||||
logger.info("")
|
||||
|
|
@ -341,19 +398,18 @@ def process_campaign(campaign, dam, box, db, notifier, config):
|
|||
os.makedirs("temp")
|
||||
|
||||
with open(csv_path, 'w', newline='') as csvfile:
|
||||
fieldnames = ['Filename', 'Tracking ID', 'Campaign Number']
|
||||
fieldnames = ['Filename', 'Tracking ID', 'Campaign Number', 'Status']
|
||||
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
|
||||
|
||||
writer.writeheader()
|
||||
for asset in processed_assets:
|
||||
# 2024-03-22: Clean filename request (remove tracking ID)
|
||||
# Assuming tracking ID is at the end or we just want the asset_name
|
||||
clean_name = asset['asset_name'] # asset_name from db.store_master_asset is typically used
|
||||
|
||||
clean_name = asset['asset_name']
|
||||
|
||||
writer.writerow({
|
||||
'Filename': clean_name,
|
||||
'Tracking ID': asset['tracking_id'],
|
||||
'Campaign Number': campaign_number
|
||||
'Campaign Number': campaign_number,
|
||||
'Status': 'Existing' if asset.get('is_existing') else 'New'
|
||||
})
|
||||
|
||||
logger.info("Generated CSV report: {}".format(csv_path))
|
||||
|
|
@ -372,6 +428,8 @@ def process_campaign(campaign, dam, box, db, notifier, config):
|
|||
'campaign_id': campaign_id,
|
||||
'campaign_number': campaign_number,
|
||||
'asset_count': len(processed_assets),
|
||||
'new_asset_count': len(new_assets),
|
||||
'existing_asset_count': len(existing_assets),
|
||||
'processed_assets': processed_assets
|
||||
},
|
||||
attachments=attachments
|
||||
|
|
|
|||
|
|
@ -57,7 +57,7 @@ logging.basicConfig(
|
|||
|
||||
logger = logging.getLogger('A2toA3')
|
||||
|
||||
def process_box_file(file_info, dam, box, db, parser, mvp_extractor, config, keep_files=False, dryrun=False):
|
||||
def process_box_file(file_info, dam, box, db, parser, mvp_extractor, config, notifier, keep_files=False, dryrun=False):
|
||||
"""
|
||||
Process a single file from Box folder
|
||||
|
||||
|
|
@ -93,11 +93,42 @@ def process_box_file(file_info, dam, box, db, parser, mvp_extractor, config, kee
|
|||
if subfolder_path:
|
||||
logger.info("From Box subfolder: {} -> will create in DAM".format(subfolder_path))
|
||||
|
||||
# 2. Load master metadata from database
|
||||
master_asset = db.get_master_asset(tracking_id)
|
||||
# 2. Load master metadata from database (support multiple tracking IDs in PPR)
|
||||
tracking_ids = parsed.get('tracking_ids', [tracking_id]) # Get all IDs or fallback to single
|
||||
has_multiple_masters = parsed.get('has_multiple_masters', False)
|
||||
|
||||
# CHECK: Warn if Master Tracking ID is used (starts with M)
|
||||
if tracking_id.upper().startswith('M'):
|
||||
# Load all master assets (PPR: multiple, PROD: single)
|
||||
master_assets = []
|
||||
master_opentext_ids = []
|
||||
|
||||
if has_multiple_masters:
|
||||
logger.info("PPR - Multiple master assets detected: {}".format(', '.join(tracking_ids)))
|
||||
for tid in tracking_ids:
|
||||
master = db.get_master_asset(tid)
|
||||
if not master:
|
||||
logger.warning("Master asset not found for tracking ID: {} - skipping".format(tid))
|
||||
continue
|
||||
master_assets.append(master)
|
||||
master_opentext_ids.append(master['opentext_id'])
|
||||
|
||||
if not master_assets:
|
||||
raise ValueError("No master assets found for tracking IDs: {}".format(', '.join(tracking_ids)))
|
||||
|
||||
# Use first master for metadata inheritance
|
||||
master_asset = master_assets[0]
|
||||
logger.info("Using primary master {} for metadata, linking {} total masters".format(
|
||||
tracking_ids[0], len(master_assets)))
|
||||
else:
|
||||
# Single master (backward compatible)
|
||||
master_asset = db.get_master_asset(tracking_id)
|
||||
if not master_asset:
|
||||
# Will check below
|
||||
master_asset = None
|
||||
else:
|
||||
master_opentext_ids = [master_asset['opentext_id']]
|
||||
|
||||
# CHECK: Warn if Master Tracking ID is used (starts with uppercase M)
|
||||
if tracking_id.startswith('M'):
|
||||
logger.warning("Detected Master Tracking ID in Version/Derivative upload folder: {}".format(tracking_id))
|
||||
|
||||
# Send notification
|
||||
|
|
@ -141,10 +172,8 @@ def process_box_file(file_info, dam, box, db, parser, mvp_extractor, config, kee
|
|||
if not master_asset:
|
||||
raise ValueError("No master asset for tracking ID: {}".format(tracking_id))
|
||||
|
||||
# 3. Get CreativeX score from database (lookup by original Box filename)
|
||||
# The PDF contains the filename field with the full name (job + tracking ID)
|
||||
# So we lookup using the original filename from Box, not the stripped version
|
||||
creativex_data = db.get_creativex_score_by_filename(filename)
|
||||
# 3. Get CreativeX score from database (lookup by filename, fallback to tracking ID)
|
||||
creativex_data = db.get_creativex_score_by_filename(filename, tracking_id=tracking_id)
|
||||
|
||||
# Build box_metadata dict (for compatibility with existing code)
|
||||
if creativex_data:
|
||||
|
|
@ -191,7 +220,8 @@ def process_box_file(file_info, dam, box, db, parser, mvp_extractor, config, kee
|
|||
parsed_filename=parsed,
|
||||
box_metadata=box_metadata, # Pass CreativeX data from database
|
||||
tracking_mode=tracking_mode, # Pass tracking mode for folder-only handling
|
||||
master_opentext_id=master_asset['opentext_id'] # Pass master DAM ID for derivative tracking
|
||||
master_opentext_id=master_asset['opentext_id'], # Primary master DAM ID
|
||||
master_opentext_ids=master_opentext_ids # All master IDs (PPR: multiple, PROD: single)
|
||||
)
|
||||
|
||||
# DRYRUN MODE: Display full asset representation and exit
|
||||
|
|
@ -215,6 +245,20 @@ def process_box_file(file_info, dam, box, db, parser, mvp_extractor, config, kee
|
|||
logger.info(" Score: {}".format(box_metadata.get('score')))
|
||||
logger.info(" URL: {}".format(box_metadata.get('url')))
|
||||
logger.info("")
|
||||
|
||||
# PPR ONLY: Register master asset IDs in lookup domain (even in dryrun for testing)
|
||||
# This API call is safe - it only adds values to the lookup table, doesn't create assets
|
||||
if master_opentext_ids:
|
||||
logger.info("PPR Domain Registration Test:")
|
||||
registration_result = dam.register_master_asset_ids_for_ppr(master_opentext_ids)
|
||||
if registration_result.get('skipped'):
|
||||
logger.info(" Skipped (not PPR environment)")
|
||||
else:
|
||||
logger.info(" Registered: {}".format(registration_result.get('registered_ids', [])))
|
||||
if registration_result.get('failed_ids'):
|
||||
logger.info(" Failed: {}".format(registration_result.get('failed_ids', [])))
|
||||
logger.info("")
|
||||
|
||||
logger.info("DRYRUN: No upload performed, file kept in Box")
|
||||
logger.info("=" * 80)
|
||||
|
||||
|
|
@ -248,6 +292,11 @@ def process_box_file(file_info, dam, box, db, parser, mvp_extractor, config, kee
|
|||
)
|
||||
logger.info("Will upload to: 01. Final Assets/{}".format(subfolder_path))
|
||||
|
||||
# PPR ONLY: Register master asset IDs in lookup domain before upload
|
||||
# OpenText API requires domain values to exist before they can be used in asset creation
|
||||
if master_opentext_ids:
|
||||
dam.register_master_asset_ids_for_ppr(master_opentext_ids)
|
||||
|
||||
upload_result = dam.upload_asset(
|
||||
file_path=clean_temp_file,
|
||||
folder_id=upload_folder_id,
|
||||
|
|
@ -347,7 +396,7 @@ def main():
|
|||
box = BoxClient(config, root_folder_id=config['box'].get('root_folder_a2_a3'))
|
||||
db = Database(config)
|
||||
notifier = Notifier(config)
|
||||
parser = FilenameParser()
|
||||
parser = FilenameParser(dam_base_url=dam.base_url) # Pass DAM URL for environment detection
|
||||
mvp_extractor = MetadataExtractorMVP(field_mappings)
|
||||
|
||||
# Test connections
|
||||
|
|
@ -426,7 +475,7 @@ def main():
|
|||
logger.info("Processing file {}/{}".format(idx, len(valid_files)))
|
||||
logger.info("=" * 60)
|
||||
|
||||
result = process_box_file(file_info, dam, box, db, parser, mvp_extractor, config, keep_files=args.keep_files, dryrun=args.dryrun)
|
||||
result = process_box_file(file_info, dam, box, db, parser, mvp_extractor, config, notifier, keep_files=args.keep_files, dryrun=args.dryrun)
|
||||
|
||||
if result['success']:
|
||||
successful_files.append(result)
|
||||
|
|
|
|||
|
|
@ -1201,6 +1201,156 @@ class DAMClient:
|
|||
mime_type, _ = mimetypes.guess_type(file_path)
|
||||
return mime_type or 'application/octet-stream'
|
||||
|
||||
def register_master_asset_id_domain_value(self, master_asset_id):
|
||||
"""
|
||||
Register a master asset ID in the FERRERO_MASTER_ASSET_ID lookup domain.
|
||||
Required in PPR environment before using the ID in asset creation.
|
||||
|
||||
The OpenText API does not support creating new domain values during asset
|
||||
creation, so this must be called before the create asset API.
|
||||
|
||||
Args:
|
||||
master_asset_id: The master asset ID to register
|
||||
|
||||
Returns:
|
||||
dict with success, http_code, and optional error
|
||||
"""
|
||||
# Only for PPR environment
|
||||
if 'ppr' not in self.base_url.lower():
|
||||
return {'success': True, 'skipped': True, 'reason': 'Not PPR environment'}
|
||||
|
||||
try:
|
||||
payload = {
|
||||
"domain_value_resource": {
|
||||
"domain_value": {
|
||||
"description": master_asset_id,
|
||||
"display_value": master_asset_id,
|
||||
"field_value": {
|
||||
"type": "string",
|
||||
"value": master_asset_id
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
logger.info("PPR: Registering master asset ID '{}' in lookup domain...".format(master_asset_id))
|
||||
|
||||
response = self._make_api_request(
|
||||
'POST',
|
||||
"{}/v6/lookupdomains/FERRERO_MASTER_ASSET_ID/lookupvalues".format(self.base_url),
|
||||
json=payload,
|
||||
headers={
|
||||
'Content-Type': 'application/json',
|
||||
'Accept': 'application/json'
|
||||
}
|
||||
)
|
||||
|
||||
# Success cases
|
||||
if response.status_code in [200, 201, 202]:
|
||||
logger.info("PPR: Master asset ID '{}' registered successfully".format(master_asset_id))
|
||||
return {
|
||||
'success': True,
|
||||
'http_code': response.status_code,
|
||||
'already_existed': False
|
||||
}
|
||||
|
||||
# Already exists - OpenText returns 409 OR 500 with "duplicate code" message
|
||||
if response.status_code == 409:
|
||||
logger.info("PPR: Master asset ID '{}' already exists in lookup domain".format(master_asset_id))
|
||||
return {
|
||||
'success': True,
|
||||
'http_code': response.status_code,
|
||||
'already_existed': True
|
||||
}
|
||||
|
||||
# Check for duplicate error in 500 response (OpenText quirk)
|
||||
if response.status_code == 500:
|
||||
try:
|
||||
error_data = response.json()
|
||||
error_msg = error_data.get('exception_body', {}).get('message', '')
|
||||
if 'duplicate' in error_msg.lower():
|
||||
logger.info("PPR: Master asset ID '{}' already exists in lookup domain".format(master_asset_id))
|
||||
return {
|
||||
'success': True,
|
||||
'http_code': response.status_code,
|
||||
'already_existed': True
|
||||
}
|
||||
except:
|
||||
pass
|
||||
|
||||
# Actual failure
|
||||
error_msg = "Failed to register master asset ID '{}': HTTP {} - {}".format(
|
||||
master_asset_id,
|
||||
response.status_code,
|
||||
response.text[:200] if response.text else 'No response'
|
||||
)
|
||||
logger.warning(error_msg)
|
||||
return {
|
||||
'success': False,
|
||||
'http_code': response.status_code,
|
||||
'error': error_msg
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
error_msg = "Exception registering master asset ID '{}': {}".format(master_asset_id, str(e))
|
||||
logger.error(error_msg)
|
||||
return {
|
||||
'success': False,
|
||||
'error': error_msg
|
||||
}
|
||||
|
||||
def register_master_asset_ids_for_ppr(self, master_asset_ids):
|
||||
"""
|
||||
Register all master asset IDs in the lookup domain (PPR only).
|
||||
Call this before creating an asset that references these IDs.
|
||||
|
||||
The OpenText DAM API does not support creating new domain values during
|
||||
asset creation. In PPR, we must first add each master asset ID to the
|
||||
FERRERO_MASTER_ASSET_ID domain value table before the create asset call.
|
||||
|
||||
Args:
|
||||
master_asset_ids: List of master asset IDs to register
|
||||
|
||||
Returns:
|
||||
dict with success, registered_ids, failed_ids
|
||||
"""
|
||||
# Only for PPR environment
|
||||
if 'ppr' not in self.base_url.lower():
|
||||
logger.debug("Not PPR environment - skipping master asset ID domain registration")
|
||||
return {'success': True, 'skipped': True}
|
||||
|
||||
if not master_asset_ids:
|
||||
return {'success': True, 'registered_ids': [], 'failed_ids': []}
|
||||
|
||||
logger.info("=" * 60)
|
||||
logger.info("PPR: Registering {} master asset ID(s) in lookup domain".format(len(master_asset_ids)))
|
||||
logger.info(" IDs: {}".format(', '.join(master_asset_ids)))
|
||||
logger.info("=" * 60)
|
||||
|
||||
registered = []
|
||||
failed = []
|
||||
|
||||
for master_id in master_asset_ids:
|
||||
result = self.register_master_asset_id_domain_value(master_id)
|
||||
if result.get('success'):
|
||||
registered.append(master_id)
|
||||
else:
|
||||
failed.append({'id': master_id, 'error': result.get('error')})
|
||||
|
||||
logger.info("PPR: Domain registration complete - {}/{} succeeded".format(
|
||||
len(registered), len(master_asset_ids)))
|
||||
|
||||
if failed:
|
||||
logger.warning("PPR: Failed to register: {}".format(
|
||||
', '.join([f['id'] for f in failed])))
|
||||
|
||||
# Return success even if some failed (better to try the upload and see)
|
||||
return {
|
||||
'success': len(failed) == 0,
|
||||
'registered_ids': registered,
|
||||
'failed_ids': failed
|
||||
}
|
||||
|
||||
def get_or_create_subfolder_path(self, base_folder_id, subfolder_path):
|
||||
"""
|
||||
Create or find subfolder structure in DAM matching Box structure
|
||||
|
|
|
|||
|
|
@ -256,18 +256,49 @@ class Database:
|
|||
# Parse JSONB as dict
|
||||
full_metadata = row[3] if isinstance(row[3], dict) else json.loads(row[3])
|
||||
|
||||
# Parse Box info from description
|
||||
box_info = self.parse_box_info_from_description(row[4])
|
||||
|
||||
return {
|
||||
'tracking_id': row[0],
|
||||
'opentext_id': row[1],
|
||||
'upload_directory': row[2],
|
||||
'full_metadata': full_metadata,
|
||||
'description': row[4]
|
||||
'description': row[4],
|
||||
'box_file_id': box_info.get('box_file_id'),
|
||||
'box_url': box_info.get('box_url')
|
||||
}
|
||||
|
||||
finally:
|
||||
cursor.close()
|
||||
self.put_connection(conn)
|
||||
|
||||
@staticmethod
|
||||
def parse_box_info_from_description(description):
|
||||
"""
|
||||
Parse Box file ID and URL from master asset description field.
|
||||
|
||||
Description format:
|
||||
Box File ID: {id}
|
||||
Box URL: {url}
|
||||
DAM Asset ID: {opentext_id}
|
||||
|
||||
Returns:
|
||||
dict with box_file_id and box_url (None if not found)
|
||||
"""
|
||||
result = {'box_file_id': None, 'box_url': None}
|
||||
if not description:
|
||||
return result
|
||||
|
||||
for line in description.split('\n'):
|
||||
line = line.strip()
|
||||
if line.startswith('Box File ID:'):
|
||||
result['box_file_id'] = line.split(':', 1)[1].strip()
|
||||
elif line.startswith('Box URL:'):
|
||||
result['box_url'] = line.split(':', 1)[1].strip()
|
||||
|
||||
return result
|
||||
|
||||
def check_campaign_upload_complete(self, campaign_id):
|
||||
"""
|
||||
Check if ALL master assets for a campaign have been uploaded
|
||||
|
|
@ -519,6 +550,157 @@ class Database:
|
|||
cursor.close()
|
||||
self.put_connection(conn)
|
||||
|
||||
def get_a1_retry_status(self, campaign_id):
|
||||
"""
|
||||
Get A1 retry status for campaign
|
||||
|
||||
Args:
|
||||
campaign_id: DAM campaign folder ID
|
||||
|
||||
Returns:
|
||||
dict with retry_count, last_retry_at, permanently_failed, failure_reason
|
||||
Returns None if campaign not found
|
||||
"""
|
||||
conn = self.get_connection()
|
||||
try:
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("""
|
||||
SELECT a1_retry_count, a1_last_retry_at,
|
||||
a1_permanently_failed, a1_failure_reason
|
||||
FROM campaign_status
|
||||
WHERE campaign_id = %s
|
||||
""", (campaign_id,))
|
||||
|
||||
row = cursor.fetchone()
|
||||
|
||||
if row:
|
||||
return {
|
||||
'retry_count': row[0] or 0,
|
||||
'last_retry_at': row[1],
|
||||
'permanently_failed': row[2] or False,
|
||||
'failure_reason': row[3]
|
||||
}
|
||||
else:
|
||||
return None
|
||||
|
||||
finally:
|
||||
cursor.close()
|
||||
self.put_connection(conn)
|
||||
|
||||
def increment_a1_retry(self, campaign_id, campaign_number, campaign_name, reason):
|
||||
"""
|
||||
Increment A1 retry counter and mark as permanently failed if max attempts reached
|
||||
|
||||
Args:
|
||||
campaign_id: DAM campaign folder ID
|
||||
campaign_number: Campaign number (e.g., C000000078)
|
||||
campaign_name: Campaign name
|
||||
reason: Description of failure (e.g., "No master assets found")
|
||||
|
||||
Returns:
|
||||
dict with success, retry_count, permanently_failed
|
||||
"""
|
||||
conn = self.get_connection()
|
||||
try:
|
||||
cursor = conn.cursor()
|
||||
|
||||
# Maximum retry attempts before marking as permanently failed
|
||||
MAX_RETRIES = 3
|
||||
|
||||
# Get current retry count
|
||||
cursor.execute("""
|
||||
SELECT a1_retry_count FROM campaign_status
|
||||
WHERE campaign_id = %s
|
||||
""", (campaign_id,))
|
||||
|
||||
row = cursor.fetchone()
|
||||
current_count = (row[0] or 0) if row else 0
|
||||
new_count = current_count + 1
|
||||
is_permanently_failed = new_count >= MAX_RETRIES
|
||||
|
||||
# Insert or update campaign status with retry tracking
|
||||
cursor.execute("""
|
||||
INSERT INTO campaign_status (
|
||||
campaign_id, campaign_number, campaign_name,
|
||||
live_campaign, status, webhook_sent,
|
||||
a1_retry_count, a1_last_retry_at,
|
||||
a1_permanently_failed, a1_failure_reason
|
||||
) VALUES (%s, %s, %s, 'NO', 'A1', FALSE, %s, CURRENT_TIMESTAMP, %s, %s)
|
||||
ON CONFLICT (campaign_id) DO UPDATE SET
|
||||
a1_retry_count = EXCLUDED.a1_retry_count,
|
||||
a1_last_retry_at = EXCLUDED.a1_last_retry_at,
|
||||
a1_permanently_failed = EXCLUDED.a1_permanently_failed,
|
||||
a1_failure_reason = EXCLUDED.a1_failure_reason,
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
""", (
|
||||
campaign_id,
|
||||
campaign_number,
|
||||
campaign_name,
|
||||
new_count,
|
||||
is_permanently_failed,
|
||||
reason if is_permanently_failed else None
|
||||
))
|
||||
|
||||
conn.commit()
|
||||
|
||||
logger.info("A1 retry tracking: Campaign {} - Attempt {}/{} (Permanently Failed: {})".format(
|
||||
campaign_number, new_count, MAX_RETRIES, is_permanently_failed
|
||||
))
|
||||
|
||||
return {
|
||||
'success': True,
|
||||
'retry_count': new_count,
|
||||
'permanently_failed': is_permanently_failed
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
conn.rollback()
|
||||
logger.error("Failed to increment A1 retry: {}".format(str(e)))
|
||||
return {'success': False, 'error': str(e)}
|
||||
|
||||
finally:
|
||||
cursor.close()
|
||||
self.put_connection(conn)
|
||||
|
||||
def reset_a1_retry(self, campaign_id):
|
||||
"""
|
||||
Reset A1 retry tracking for campaign (used when campaign is fixed manually)
|
||||
|
||||
Args:
|
||||
campaign_id: DAM campaign folder ID
|
||||
|
||||
Returns:
|
||||
dict with success boolean
|
||||
"""
|
||||
conn = self.get_connection()
|
||||
try:
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("""
|
||||
UPDATE campaign_status
|
||||
SET a1_retry_count = 0,
|
||||
a1_last_retry_at = NULL,
|
||||
a1_permanently_failed = FALSE,
|
||||
a1_failure_reason = NULL,
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
WHERE campaign_id = %s
|
||||
""", (campaign_id,))
|
||||
|
||||
conn.commit()
|
||||
logger.info("Reset A1 retry tracking for campaign: {}".format(campaign_id))
|
||||
|
||||
return {'success': True}
|
||||
|
||||
except Exception as e:
|
||||
conn.rollback()
|
||||
logger.error("Failed to reset A1 retry: {}".format(str(e)))
|
||||
return {'success': False, 'error': str(e)}
|
||||
|
||||
finally:
|
||||
cursor.close()
|
||||
self.put_connection(conn)
|
||||
|
||||
def check_campaign_processed(self, campaign_id):
|
||||
"""
|
||||
Check if campaign has already been processed (webhook sent)
|
||||
|
|
@ -693,15 +875,18 @@ class Database:
|
|||
cursor.close()
|
||||
self.put_connection(conn)
|
||||
|
||||
def get_creativex_score_by_filename(self, filename):
|
||||
def get_creativex_score_by_filename(self, filename, tracking_id=None):
|
||||
"""
|
||||
Get CreativeX score data by filename
|
||||
|
||||
Performs extension-agnostic lookup: if exact filename not found,
|
||||
tries common video/image extensions (.mp4, .jpg, .png, .mov, etc.)
|
||||
If still not found and tracking_id provided, falls back to LIKE search
|
||||
on tracking ID (handles mismatched naming from CreativeX PDFs).
|
||||
|
||||
Args:
|
||||
filename: Filename to search for
|
||||
tracking_id: Optional tracking ID for fallback lookup
|
||||
|
||||
Returns:
|
||||
dict with creativex data or None if not found
|
||||
|
|
@ -748,6 +933,24 @@ class Database:
|
|||
if row:
|
||||
break # Found with alternative extension
|
||||
|
||||
# If still not found, try tracking ID fallback
|
||||
# CreativeX PDFs sometimes have different naming (extra text, stripped hyphens)
|
||||
# but tracking ID is always consistent
|
||||
if not row and tracking_id:
|
||||
cursor.execute("""
|
||||
SELECT filename, creativex_id, creativex_url, quality_score,
|
||||
box_file_id, full_extraction_data, extracted_at
|
||||
FROM creativex_scores
|
||||
WHERE filename LIKE %s AND status = 'active'
|
||||
ORDER BY extracted_at DESC
|
||||
LIMIT 1
|
||||
""", ('%' + tracking_id + '%',))
|
||||
|
||||
row = cursor.fetchone()
|
||||
if row:
|
||||
logger.info("CreativeX: Found score via tracking ID fallback '{}' -> {}".format(
|
||||
tracking_id, row[0]))
|
||||
|
||||
if not row:
|
||||
return None
|
||||
|
||||
|
|
|
|||
|
|
@ -15,10 +15,43 @@ class FilenameParser:
|
|||
[JOB]_[BRAND]_[SUBJECT]_[ASSET]_[DUR]_[RATIO]_[SPOT]_[COUNTRY]_[LANG]_[SOCIAL]_[TRACKING]
|
||||
|
||||
Example: 1234567_RAF_ME-MOMENT_OLV_6S_1x1_REF_GL_it_IGF_pOiJ9s
|
||||
|
||||
PPR Environment: Supports multiple tracking IDs (e.g., pOiJ9s+BqB8vo+laRJo0)
|
||||
PROD Environment: Single tracking ID only (backward compatible)
|
||||
"""
|
||||
|
||||
# Known social media platform codes
|
||||
SOCIAL_MEDIA_CODES = ['FBP', 'FBR', 'IGF', 'IGR'] # Expandable
|
||||
# Known social media platform codes (from Ferrero naming tool data.json)
|
||||
SOCIAL_MEDIA_CODES = [
|
||||
# Facebook
|
||||
'FBD', 'FGF', 'FBR', 'FRO', 'FBS', 'FBF', 'FBP', 'FIA', 'FIV',
|
||||
'FMP', 'FPF', 'FRC', 'FSE', 'FSS', 'FSV', 'FUK', 'FVF',
|
||||
# Instagram
|
||||
'IGF', 'IGE', 'IGG', 'IGT', 'IPF', 'IPR', 'IGR', 'IGO', 'IGS', 'ISH', 'IST',
|
||||
# Audience Network
|
||||
'ANC', 'ANI', 'ANR',
|
||||
# Messenger
|
||||
'MSI', 'MSS',
|
||||
# YouTube
|
||||
'YTA', 'YTB', 'YTS',
|
||||
# Other platforms
|
||||
'AMZ', 'DV3', 'GOO', 'PIN', 'SNA', 'TIK', 'TWI', 'VOD',
|
||||
]
|
||||
|
||||
def __init__(self, dam_base_url=None):
|
||||
"""
|
||||
Initialize parser with optional environment detection
|
||||
|
||||
Args:
|
||||
dam_base_url: DAM base URL for environment detection (optional)
|
||||
"""
|
||||
self.dam_base_url = dam_base_url
|
||||
self.is_ppr = self._is_ppr_environment()
|
||||
|
||||
def _is_ppr_environment(self):
|
||||
"""Check if running in PPR environment"""
|
||||
if not self.dam_base_url:
|
||||
return False
|
||||
return 'ppr.dam.ferrero.com' in self.dam_base_url.lower()
|
||||
|
||||
def parse_filename(self, filename):
|
||||
"""
|
||||
|
|
@ -178,21 +211,68 @@ class FilenameParser:
|
|||
logger.debug("Found social media: {}".format(part))
|
||||
index += 1
|
||||
|
||||
# Tracking ID: 6 alphanumeric, optionally with -N suffix
|
||||
elif re.match(r'^[a-zA-Z0-9]{6}(-N)?$', part):
|
||||
tracking = part
|
||||
tracking_mode = 'full'
|
||||
base_tracking_id = tracking
|
||||
# Tracking ID(s): 6 alphanumeric, optionally with -N suffix
|
||||
# PPR: Supports multiple IDs (e.g., "BqB8vo+SfUQ7m+laRJo0")
|
||||
# PROD: Single ID only (backward compatible)
|
||||
elif re.match(r'^[a-zA-Z0-9]{6}(-N)?(\+[a-zA-Z0-9]{6}(-N)?)*$', part):
|
||||
# Check if multiple IDs provided
|
||||
if '+' in part and self.is_ppr:
|
||||
# PPR ONLY: Parse multiple tracking IDs
|
||||
tracking_ids = []
|
||||
tracking_modes = []
|
||||
tracking_ids_with_suffix = []
|
||||
|
||||
if tracking.endswith('-N'):
|
||||
tracking_mode = 'folder_only'
|
||||
base_tracking_id = tracking[:-2] # Strip -N suffix
|
||||
logger.info("Detected folder-only tracking ID: {} (base: {})".format(tracking, base_tracking_id))
|
||||
id_parts = part.split('+')
|
||||
logger.info("PPR Environment - Multiple tracking IDs detected: {}".format(len(id_parts)))
|
||||
|
||||
for tracking in id_parts:
|
||||
tracking_mode = 'full'
|
||||
base_tracking_id = tracking
|
||||
|
||||
if tracking.endswith('-N'):
|
||||
tracking_mode = 'folder_only'
|
||||
base_tracking_id = tracking[:-2]
|
||||
logger.info("Folder-only tracking ID: {} (base: {})".format(tracking, base_tracking_id))
|
||||
|
||||
tracking_ids.append(base_tracking_id)
|
||||
tracking_modes.append(tracking_mode)
|
||||
tracking_ids_with_suffix.append(tracking)
|
||||
|
||||
# Store primary (first) for backward compatibility
|
||||
parsed['tracking_id'] = tracking_ids[0]
|
||||
parsed['tracking_mode'] = tracking_modes[0]
|
||||
parsed['tracking_id_with_suffix'] = tracking_ids_with_suffix[0]
|
||||
|
||||
# Store all IDs for multi-master support
|
||||
parsed['tracking_ids'] = tracking_ids
|
||||
parsed['tracking_modes'] = tracking_modes
|
||||
parsed['tracking_ids_with_suffix'] = tracking_ids_with_suffix
|
||||
parsed['has_multiple_masters'] = True
|
||||
|
||||
logger.info("Parsed {} tracking IDs: {}".format(len(tracking_ids), ', '.join(tracking_ids)))
|
||||
else:
|
||||
# PROD or Single ID: Use only first tracking ID
|
||||
if '+' in part:
|
||||
logger.warning("PROD Environment - Multiple tracking IDs not supported, using first ID only")
|
||||
part = part.split('+')[0] # Take only first ID
|
||||
|
||||
tracking = part
|
||||
tracking_mode = 'full'
|
||||
base_tracking_id = tracking
|
||||
|
||||
if tracking.endswith('-N'):
|
||||
tracking_mode = 'folder_only'
|
||||
base_tracking_id = tracking[:-2]
|
||||
logger.info("Folder-only tracking ID: {} (base: {})".format(tracking, base_tracking_id))
|
||||
|
||||
parsed['tracking_id'] = base_tracking_id
|
||||
parsed['tracking_mode'] = tracking_mode
|
||||
parsed['tracking_id_with_suffix'] = tracking
|
||||
parsed['tracking_ids'] = [base_tracking_id] # Single item list for compatibility
|
||||
parsed['has_multiple_masters'] = False
|
||||
|
||||
logger.debug("Found tracking ID: {}".format(tracking))
|
||||
|
||||
parsed['tracking_id'] = base_tracking_id
|
||||
parsed['tracking_mode'] = tracking_mode
|
||||
parsed['tracking_id_with_suffix'] = tracking
|
||||
logger.debug("Found tracking ID: {}".format(tracking))
|
||||
index += 1
|
||||
|
||||
# Unknown part - could be aspect ratio fallback
|
||||
|
|
@ -216,8 +296,8 @@ class FilenameParser:
|
|||
|
||||
def strip_upload_components(self, filename):
|
||||
"""
|
||||
Strip OMG Job Number and Tracking ID from filename
|
||||
Returns clean filename in V2.1 order
|
||||
Strip OMG Job Number from front and Tracking ID from back of filename.
|
||||
Keeps everything else as-is (including social media codes, DV3, etc.)
|
||||
|
||||
Args:
|
||||
filename: Original filename
|
||||
|
|
@ -226,40 +306,23 @@ class FilenameParser:
|
|||
Clean filename for upload (no job number, no tracking ID)
|
||||
|
||||
Example:
|
||||
Input: 1234567_RAF_TEST_OLV_6S_1x1_REF_GL_it_IGF_abc123.mp4
|
||||
Output: RAF_TEST_OLV_6S_1x1_REF_GL_it_IGF.mp4
|
||||
Input: 6662777_NUT_XMAS-SHARETHELOVE-GLAS_OLV_6S_16X9_PL_pl_YTA_EvQJrM.mp4
|
||||
Output: NUT_XMAS-SHARETHELOVE-GLAS_OLV_6S_16X9_PL_pl_YTA.mp4
|
||||
"""
|
||||
parsed = self.parse_filename(filename)
|
||||
import os
|
||||
|
||||
if not parsed:
|
||||
base, ext = os.path.splitext(filename)
|
||||
parts = base.split('_')
|
||||
|
||||
if len(parts) < 3:
|
||||
return filename
|
||||
|
||||
# Build clean filename in V2.1 order
|
||||
# [BRAND]_[SUBJECT]_[ASSET]_[DUR]_[RATIO]_[SPOT]_[COUNTRY]_[LANG]_[SOCIAL]
|
||||
clean_parts = []
|
||||
# Strip job number from front (digits only)
|
||||
if parts[0].isdigit():
|
||||
parts = parts[1:]
|
||||
|
||||
if parsed['brand_code']:
|
||||
clean_parts.append(parsed['brand_code'])
|
||||
if parsed['subject_title']:
|
||||
clean_parts.append(parsed['subject_title'])
|
||||
if parsed['asset_type']:
|
||||
clean_parts.append(parsed['asset_type'])
|
||||
if parsed['seconds']:
|
||||
clean_parts.append(parsed['seconds'] + 'S')
|
||||
if parsed['aspect_ratio']:
|
||||
clean_parts.append(parsed['aspect_ratio'])
|
||||
if parsed['spot_version']:
|
||||
clean_parts.append(parsed['spot_version'])
|
||||
if parsed['country_code']:
|
||||
clean_parts.append(parsed['country_code'])
|
||||
if parsed['language_code']:
|
||||
clean_parts.append(parsed['language_code'])
|
||||
if parsed['social_media_version']:
|
||||
clean_parts.append(parsed['social_media_version'])
|
||||
# Strip tracking ID(s) from back (6 alphanumeric chars, optionally with +joined IDs or -N suffix)
|
||||
if parts and re.match(r'^[a-zA-Z0-9]{6}(-N)?(\+[a-zA-Z0-9]{6}(-N)?)*$', parts[-1]):
|
||||
parts = parts[:-1]
|
||||
|
||||
clean_filename = '_'.join(clean_parts)
|
||||
|
||||
if parsed['extension']:
|
||||
clean_filename += parsed['extension']
|
||||
|
||||
return clean_filename
|
||||
return '_'.join(parts) + ext
|
||||
|
|
|
|||
|
|
@ -5,6 +5,8 @@ Compatible with Python 3.6+
|
|||
"""
|
||||
|
||||
import logging
|
||||
import json
|
||||
import copy
|
||||
from datetime import datetime, timedelta
|
||||
import os
|
||||
from shared.config_loader import load_country_code_mappings
|
||||
|
|
@ -23,6 +25,7 @@ class MetadataExtractorMVP:
|
|||
self.filename_updates = field_mappings.get('filename_updates', {})
|
||||
self.forced_values = field_mappings.get('forced_values', {})
|
||||
self.defaults = field_mappings.get('defaults', {})
|
||||
self.asset_type_overrides = field_mappings.get('asset_type_overrides', {})
|
||||
|
||||
# Load country code mappings (ISO -> DAM codes)
|
||||
self.country_mappings = load_country_code_mappings()
|
||||
|
|
@ -34,6 +37,22 @@ class MetadataExtractorMVP:
|
|||
if self.asset_type_mappings:
|
||||
logger.info("Loaded {} asset type mappings (3-letter->DAM)".format(len(self.asset_type_mappings)))
|
||||
|
||||
# Load asset representation template for folder-only mode
|
||||
self.template_fields = self._load_asset_representation_template()
|
||||
if self.template_fields:
|
||||
logger.info("Loaded asset representation template with {} fields".format(len(self.template_fields)))
|
||||
|
||||
def _load_asset_representation_template(self):
|
||||
"""Load the asset representation template JSON for folder-only mode"""
|
||||
template_path = 'config/asset_representation_template.json'
|
||||
try:
|
||||
with open(template_path, 'r') as f:
|
||||
data = json.load(f)
|
||||
return data['asset_resource']['asset']['metadata']['metadata_element_list']
|
||||
except Exception as e:
|
||||
logger.warning("Could not load asset representation template: {}".format(str(e)))
|
||||
return []
|
||||
|
||||
def extract_mvp_fields(self, master_metadata):
|
||||
"""
|
||||
Extract only MVP fields from full master metadata
|
||||
|
|
@ -94,7 +113,7 @@ class MetadataExtractorMVP:
|
|||
|
||||
return extracted_fields
|
||||
|
||||
def build_mvp_asset_representation(self, master_metadata, clean_filename, parsed_filename, box_metadata=None, tracking_mode='full', master_opentext_id=None):
|
||||
def build_mvp_asset_representation(self, master_metadata, clean_filename, parsed_filename, box_metadata=None, tracking_mode='full', master_opentext_id=None, master_opentext_ids=None):
|
||||
"""
|
||||
Build asset representation with MVP fields + updates from filename
|
||||
|
||||
|
|
@ -127,15 +146,33 @@ class MetadataExtractorMVP:
|
|||
mvp_fields = []
|
||||
mvp_fields = self._build_fields_from_filename(parsed_filename, clean_filename)
|
||||
|
||||
# Apply forced values from config (e.g., AGENCY NAME)
|
||||
# STATE is already handled in _build_fields_from_filename
|
||||
mvp_fields = self._apply_forced_values(mvp_fields)
|
||||
|
||||
# Add missing MVP fields with defaults (both modes)
|
||||
mvp_fields = self._add_missing_fields(mvp_fields, parsed_filename)
|
||||
|
||||
# Add empty required fields that DAM expects (even if empty) - folder-only mode needs these
|
||||
mvp_fields = self._add_empty_required_fields(mvp_fields)
|
||||
|
||||
# Apply asset type overrides (e.g., EOL) - takes final precedence over forced values/defaults
|
||||
mvp_fields = self._apply_asset_type_overrides(mvp_fields, parsed_filename)
|
||||
|
||||
# Update CreativeX fields from Box metadata if provided
|
||||
if box_metadata:
|
||||
mvp_fields = self._update_creativex_fields(mvp_fields, box_metadata)
|
||||
|
||||
# Add Master Asset ID field if provided (derivative tracking)
|
||||
if master_opentext_id:
|
||||
# Add MASTERASSETIDS field with all master IDs
|
||||
# Priority: Use master_opentext_ids if provided (multiple IDs), otherwise fall back to single master_opentext_id
|
||||
if master_opentext_ids and len(master_opentext_ids) > 0:
|
||||
mvp_fields = self._add_master_asset_ids_field(mvp_fields, master_opentext_ids)
|
||||
if len(master_opentext_ids) > 1:
|
||||
logger.info("PPR - Added MASTERASSETIDS field with {} master IDs".format(len(master_opentext_ids)))
|
||||
else:
|
||||
logger.info("Added MASTERASSETIDS field with 1 master ID")
|
||||
elif master_opentext_id:
|
||||
# Fallback to single master ID if master_opentext_ids not provided
|
||||
mvp_fields = self._add_master_asset_id_field(mvp_fields, master_opentext_id)
|
||||
logger.info("Added Master Asset ID field: {}".format(master_opentext_id))
|
||||
|
||||
|
|
@ -190,8 +227,28 @@ class MetadataExtractorMVP:
|
|||
# Update the field
|
||||
for field in mvp_fields:
|
||||
if field.get('id') == field_id:
|
||||
self._set_field_value(field, value)
|
||||
logger.info("Updated {} from filename: {}".format(field_id, value))
|
||||
# For tabular fields (like MAIN_LANGUAGES), update the 'values' array
|
||||
# The DAM reads from 'values' (plural), not 'value' (singular)
|
||||
if field.get('type') == 'com.artesia.metadata.MetadataTableField' or 'values' in field:
|
||||
field['values'] = [
|
||||
{
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'is_locked': False,
|
||||
'value': {
|
||||
'expired_value': False,
|
||||
'field_value': {
|
||||
'type': 'string',
|
||||
'value': value
|
||||
},
|
||||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
}
|
||||
]
|
||||
logger.info("Updated tabular field {} values array from filename: {}".format(field_id, value))
|
||||
else:
|
||||
self._set_field_value(field, value)
|
||||
logger.info("Updated {} from filename: {}".format(field_id, value))
|
||||
break
|
||||
|
||||
# Apply country code mapping (ISO -> DAM codes)
|
||||
|
|
@ -268,6 +325,88 @@ class MetadataExtractorMVP:
|
|||
|
||||
return mvp_fields
|
||||
|
||||
def _apply_asset_type_overrides(self, mvp_fields, parsed_filename):
|
||||
"""
|
||||
Apply asset type overrides when a matching asset type (e.g., EOL) is detected in the filename.
|
||||
These overrides take final precedence over forced values and defaults.
|
||||
|
||||
Args:
|
||||
mvp_fields: List of MVP field objects
|
||||
parsed_filename: Parsed filename dict (must contain 'asset_type' key)
|
||||
|
||||
Returns:
|
||||
Updated mvp_fields list
|
||||
"""
|
||||
if not parsed_filename:
|
||||
return mvp_fields
|
||||
|
||||
asset_type = parsed_filename.get('asset_type')
|
||||
if not asset_type:
|
||||
return mvp_fields
|
||||
|
||||
overrides = self.asset_type_overrides.get(asset_type)
|
||||
if not overrides:
|
||||
return mvp_fields
|
||||
|
||||
logger.info("Applying {} asset type overrides for '{}'".format(len(overrides), asset_type))
|
||||
|
||||
for field_id, override_value in overrides.items():
|
||||
# Empty string means remove the field entirely
|
||||
if override_value == '':
|
||||
before_count = len(mvp_fields)
|
||||
mvp_fields = [f for f in mvp_fields if f.get('id') != field_id]
|
||||
if len(mvp_fields) < before_count:
|
||||
logger.info("Asset type override: removed field {}".format(field_id))
|
||||
else:
|
||||
logger.debug("Asset type override: field {} not present (nothing to remove)".format(field_id))
|
||||
continue
|
||||
|
||||
field_found = False
|
||||
for field in mvp_fields:
|
||||
if field.get('id') == field_id:
|
||||
field_found = True
|
||||
# For tabular fields (like MAIN_LANGUAGES), update both 'value' and 'values'
|
||||
if field.get('type') == 'com.artesia.metadata.MetadataTableField' or 'values' in field:
|
||||
domain_value_obj = {
|
||||
'type': 'com.artesia.metadata.DomainValue',
|
||||
'field_value': {'type': 'string', 'value': override_value},
|
||||
'display_value': override_value,
|
||||
'expired_value': False,
|
||||
'active_to': '',
|
||||
'active_from': ''
|
||||
}
|
||||
field['value'] = {
|
||||
'value': domain_value_obj,
|
||||
'is_locked': False,
|
||||
'domain_value': True,
|
||||
'cascading_domain_value': False
|
||||
}
|
||||
field['values'] = [
|
||||
{
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'is_locked': False,
|
||||
'value': {
|
||||
'expired_value': False,
|
||||
'field_value': {
|
||||
'type': 'string',
|
||||
'value': override_value
|
||||
},
|
||||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
}
|
||||
]
|
||||
logger.info("Asset type override: {} = {} (tabular)".format(field_id, override_value))
|
||||
else:
|
||||
self._set_field_value(field, override_value)
|
||||
logger.info("Asset type override: {} = {}".format(field_id, override_value))
|
||||
break
|
||||
|
||||
if not field_found:
|
||||
logger.warning("Asset type override field '{}' not found in MVP fields - skipping".format(field_id))
|
||||
|
||||
return mvp_fields
|
||||
|
||||
def _add_missing_fields(self, mvp_fields, parsed_filename):
|
||||
"""Add missing MVP fields from filename or defaults"""
|
||||
field_ids = [f.get('id') for f in mvp_fields]
|
||||
|
|
@ -278,29 +417,82 @@ class MetadataExtractorMVP:
|
|||
language = parsed_filename['language_code'].upper()
|
||||
logger.info("Adding MAIN_LANGUAGES: {}".format(language))
|
||||
|
||||
domain_value_obj = {
|
||||
'type': 'com.artesia.metadata.DomainValue',
|
||||
'field_value': {'type': 'string', 'value': language},
|
||||
'display_value': language,
|
||||
'expired_value': False,
|
||||
'active_to': '',
|
||||
'active_from': ''
|
||||
}
|
||||
mvp_fields.append({
|
||||
'id': 'MAIN_LANGUAGES',
|
||||
'name': 'MAIN LANGUAGES',
|
||||
'parent_table_id': 'FERRERO.TABULAR.FIELD.MAIN LANGUAGES',
|
||||
'type': 'com.artesia.metadata.MetadataTableField',
|
||||
'value': {
|
||||
'value': domain_value_obj,
|
||||
'is_locked': False,
|
||||
'domain_value': True,
|
||||
'cascading_domain_value': False
|
||||
},
|
||||
'values': [
|
||||
{
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'is_locked': False,
|
||||
'value': {
|
||||
'field_value': {
|
||||
'type': 'string',
|
||||
'value': language
|
||||
},
|
||||
'expired_value': False,
|
||||
'field_value': {'type': 'string', 'value': language},
|
||||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
}
|
||||
]
|
||||
],
|
||||
'tabular': True,
|
||||
'domained': True,
|
||||
'required': True,
|
||||
'domain_id': 'FERRERO.DOMAIN.MAIN LAGUAGES_LU'
|
||||
})
|
||||
|
||||
# Add other missing fields with defaults
|
||||
field_ids = [f.get('id') for f in mvp_fields]
|
||||
|
||||
for field_id, default_value in self.defaults.items():
|
||||
if field_id in field_ids:
|
||||
# Field exists (e.g. from template) - check if value is empty and set default
|
||||
for field in mvp_fields:
|
||||
if field.get('id') == field_id:
|
||||
# Tabular fields use 'values' array - skip if already populated
|
||||
if field.get('type') == 'com.artesia.metadata.MetadataTableField':
|
||||
if field.get('values'):
|
||||
break # Already has values
|
||||
# Empty tabular - fall through to add as new below
|
||||
break
|
||||
# Regular field - check if it has an actual value set
|
||||
val = field.get('value', {})
|
||||
has_value = 'value' in val and isinstance(val.get('value'), dict) and 'value' in val['value']
|
||||
if not has_value:
|
||||
# Use DomainValue format for domained fields
|
||||
if field.get('domained', False):
|
||||
field['value'] = {
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'is_locked': False,
|
||||
'value': {
|
||||
'active_from': '',
|
||||
'active_to': '',
|
||||
'display_value': default_value,
|
||||
'expired_value': False,
|
||||
'field_value': {'type': 'string', 'value': default_value},
|
||||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
}
|
||||
else:
|
||||
field['value'] = {'value': {'type': 'string', 'value': default_value}}
|
||||
logger.info("Set default on template field {} = {}".format(field_id, default_value))
|
||||
break
|
||||
continue
|
||||
|
||||
if field_id not in field_ids:
|
||||
logger.info("Adding {} with default: {}".format(field_id, default_value))
|
||||
|
||||
|
|
@ -310,12 +502,72 @@ class MetadataExtractorMVP:
|
|||
]
|
||||
|
||||
if is_tabular:
|
||||
# Map field IDs to correct parent table IDs
|
||||
parent_table_map = {
|
||||
'FERRERO.FIELD.ASSETCOMPLIANCE': 'FERRERO.TABULAR.FIELD.ASSETCOMPLIANCE',
|
||||
'MARKETING_TAG': 'FERRERO.TABULAR.FIELD.MARKETING.TAG',
|
||||
}
|
||||
parent_table_id = parent_table_map.get(field_id, 'FERRERO.TABULAR.FIELD.' + field_id.split('.')[-1])
|
||||
|
||||
domain_value_obj = {
|
||||
'type': 'com.artesia.metadata.DomainValue',
|
||||
'field_value': {'type': 'string', 'value': default_value},
|
||||
'display_value': default_value,
|
||||
'expired_value': False,
|
||||
'active_to': '',
|
||||
'active_from': ''
|
||||
}
|
||||
mvp_fields.append({
|
||||
'id': field_id,
|
||||
'parent_table_id': 'FERRERO.TABULAR.FIELD.' + field_id.split('.')[-1],
|
||||
'parent_table_id': parent_table_id,
|
||||
'type': 'com.artesia.metadata.MetadataTableField',
|
||||
'value': {
|
||||
'value': domain_value_obj,
|
||||
'is_locked': False,
|
||||
'domain_value': True,
|
||||
'cascading_domain_value': False
|
||||
},
|
||||
'values': [
|
||||
{
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'is_locked': False,
|
||||
'value': {
|
||||
'expired_value': False,
|
||||
'field_value': {
|
||||
'type': 'string',
|
||||
'value': default_value
|
||||
},
|
||||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
}
|
||||
],
|
||||
'tabular': True,
|
||||
'domained': True
|
||||
})
|
||||
else:
|
||||
# Non-domain fields use simple value structure
|
||||
non_domain_fields = [
|
||||
'FERRERO.MARKETING.FIELD.VIDEO_POST_PROD_COMPANY',
|
||||
'FERRERO.MARKETING.FIELD.AUDIO_POST_PROD_COMPANY',
|
||||
]
|
||||
|
||||
if field_id in non_domain_fields:
|
||||
mvp_fields.append({
|
||||
'id': field_id,
|
||||
'type': 'com.artesia.metadata.MetadataField',
|
||||
'value': {
|
||||
'value': {
|
||||
'type': 'string',
|
||||
'value': default_value
|
||||
}
|
||||
}
|
||||
})
|
||||
else:
|
||||
mvp_fields.append({
|
||||
'id': field_id,
|
||||
'type': 'com.artesia.metadata.MetadataField',
|
||||
'value': {
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'value': {
|
||||
|
|
@ -326,21 +578,154 @@ class MetadataExtractorMVP:
|
|||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
}
|
||||
]
|
||||
})
|
||||
else:
|
||||
mvp_fields.append({
|
||||
'id': field_id,
|
||||
'type': 'com.artesia.metadata.MetadataField',
|
||||
'value': {
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'value': {
|
||||
'type': 'string',
|
||||
'value': default_value
|
||||
})
|
||||
|
||||
return mvp_fields
|
||||
|
||||
def _apply_forced_values(self, mvp_fields):
|
||||
"""
|
||||
Apply forced values from config to existing fields.
|
||||
For fields not yet present, adds them with DomainValue format.
|
||||
Used in folder-only mode where _update_fields is not called.
|
||||
"""
|
||||
field_ids = [f.get('id') for f in mvp_fields]
|
||||
|
||||
for field_id, forced_value in self.forced_values.items():
|
||||
if field_id in field_ids:
|
||||
# Field exists - set value with proper format based on field type
|
||||
for field in mvp_fields:
|
||||
if field.get('id') == field_id:
|
||||
if field.get('domained', False):
|
||||
field['value'] = {
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'is_locked': False,
|
||||
'value': {
|
||||
'active_from': '',
|
||||
'active_to': '',
|
||||
'display_value': forced_value,
|
||||
'expired_value': False,
|
||||
'field_value': {'type': 'string', 'value': forced_value},
|
||||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
}
|
||||
else:
|
||||
self._set_field_value(field, forced_value)
|
||||
logger.info("Forced value applied: {} = {}".format(field_id, forced_value))
|
||||
break
|
||||
else:
|
||||
# Field not present - add with DomainValue format
|
||||
mvp_fields.append({
|
||||
'id': field_id,
|
||||
'type': 'com.artesia.metadata.MetadataField',
|
||||
'value': {
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'value': {
|
||||
'field_value': {'type': 'string', 'value': forced_value},
|
||||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
})
|
||||
}
|
||||
})
|
||||
logger.info("Forced value added: {} = {}".format(field_id, forced_value))
|
||||
|
||||
return mvp_fields
|
||||
|
||||
def _add_empty_required_fields(self, mvp_fields):
|
||||
"""
|
||||
Add fields that the DAM expects to be present even if empty.
|
||||
In full-inheritance mode these come from the master asset.
|
||||
In folder-only mode they must be explicitly added.
|
||||
Only adds fields not already present.
|
||||
"""
|
||||
field_ids = [f.get('id') for f in mvp_fields]
|
||||
|
||||
# Empty value structure for domained fields with no value set
|
||||
empty_domained_value = {
|
||||
'is_locked': False,
|
||||
'domain_value': False,
|
||||
'cascading_domain_value': False
|
||||
}
|
||||
|
||||
# Fields with empty domained values
|
||||
empty_domained_fields = [
|
||||
'FERRERO.FIELD.MARKETING.FLAVOUR',
|
||||
'FERRERO.FIELD.MARKETING.SIZE',
|
||||
'FERRERO.FIELD.SUB BRAND',
|
||||
'FERRERO.MARKET.FIELD.BUYOUT',
|
||||
'FERRERO.MARKET.FIELD.FERRERO PROPERTY',
|
||||
'FERRERO.MARKET.VID_N_STAT',
|
||||
'FERRERO.MARKETING.FIELD.SPOT_VERSION',
|
||||
]
|
||||
|
||||
for field_id in empty_domained_fields:
|
||||
if field_id not in field_ids:
|
||||
mvp_fields.append({
|
||||
'id': field_id,
|
||||
'type': 'com.artesia.metadata.MetadataField',
|
||||
'value': dict(empty_domained_value)
|
||||
})
|
||||
|
||||
# Fields with empty non-domained values
|
||||
empty_plain_fields = [
|
||||
'FERRERO.MARKETING.FIELD.DIRECTOR_NAME',
|
||||
'FERRERO.MARKETING.FIELD.VID_POST_PROD_CONTACT',
|
||||
'FERRERO.MARKETING.FIELD.AUDIO_POST_PROD_CONTACT',
|
||||
'FERRERO.MARKET.FIELD.LICENSE',
|
||||
]
|
||||
|
||||
for field_id in empty_plain_fields:
|
||||
if field_id not in field_ids:
|
||||
mvp_fields.append({
|
||||
'id': field_id,
|
||||
'type': 'com.artesia.metadata.MetadataField',
|
||||
'value': {
|
||||
'is_locked': False,
|
||||
'domain_value': False,
|
||||
'cascading_domain_value': False
|
||||
}
|
||||
})
|
||||
|
||||
# Domained fields with default "No" value
|
||||
no_value_fields = [
|
||||
'FERRERO.MARKET.FIELD.IPRIGHT',
|
||||
'FERRERO.MARKET.FIELD.LICENSIN',
|
||||
]
|
||||
|
||||
for field_id in no_value_fields:
|
||||
if field_id not in field_ids:
|
||||
mvp_fields.append({
|
||||
'id': field_id,
|
||||
'type': 'com.artesia.metadata.MetadataField',
|
||||
'value': {
|
||||
'value': {
|
||||
'type': 'com.artesia.metadata.DomainValue',
|
||||
'field_value': {'type': 'string', 'value': 'No'},
|
||||
'display_value': 'No',
|
||||
'expired_value': False,
|
||||
'active_to': '',
|
||||
'active_from': ''
|
||||
},
|
||||
'is_locked': False,
|
||||
'domain_value': True,
|
||||
'cascading_domain_value': False
|
||||
}
|
||||
})
|
||||
|
||||
# Empty tabular field: Type of Video & Static Right
|
||||
if 'FERRERO.MARKET.FIELD.TYPE_VID' not in field_ids:
|
||||
mvp_fields.append({
|
||||
'id': 'FERRERO.MARKET.FIELD.TYPE_VID',
|
||||
'parent_table_id': 'FERRERO.TABULAR.VID_STAT_TYPE',
|
||||
'type': 'com.artesia.metadata.MetadataTableField',
|
||||
'values': [],
|
||||
'tabular': True,
|
||||
'domained': True
|
||||
})
|
||||
|
||||
added_count = len(mvp_fields) - len(field_ids)
|
||||
if added_count > 0:
|
||||
logger.info("Added {} empty required fields for DAM compatibility".format(added_count))
|
||||
|
||||
return mvp_fields
|
||||
|
||||
|
|
@ -415,63 +800,87 @@ class MetadataExtractorMVP:
|
|||
|
||||
def _build_fields_from_filename(self, parsed_filename, clean_filename):
|
||||
"""
|
||||
Build ALL metadata fields from parsed filename
|
||||
Used in folder-only mode (tracking ID with -N suffix)
|
||||
Build ALL metadata fields from parsed filename using the reference template.
|
||||
Used in folder-only mode (tracking ID with -N suffix).
|
||||
|
||||
Note: Uses codes directly for now. Can add lookup tables later
|
||||
for brand_code->brand_name, country_code->country_name, etc.
|
||||
Deep copies the asset representation template and populates values
|
||||
from the parsed filename. This ensures all fields have the full metadata
|
||||
structure (column_name, data_type, etc.) that the DAM API requires.
|
||||
"""
|
||||
fields = []
|
||||
if not self.template_fields:
|
||||
logger.error("No asset representation template loaded - folder-only mode cannot proceed")
|
||||
return []
|
||||
|
||||
# Deep copy the template so we don't modify the original
|
||||
fields = copy.deepcopy(self.template_fields)
|
||||
|
||||
# Build lookup for quick access
|
||||
fields_by_id = {f['id']: f for f in fields}
|
||||
|
||||
# Helper to set a domained field value with DomainValue structure
|
||||
def set_domained_value(field, value):
|
||||
field['value'] = {
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'is_locked': False,
|
||||
'value': {
|
||||
'active_from': '',
|
||||
'active_to': '',
|
||||
'display_value': value,
|
||||
'expired_value': False,
|
||||
'field_value': {'type': 'string', 'value': value},
|
||||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
}
|
||||
|
||||
# Helper to set a plain string field value
|
||||
def set_string_value(field, value):
|
||||
field['value'] = {'value': {'type': 'string', 'value': value}}
|
||||
|
||||
# --- Populate fields from filename ---
|
||||
|
||||
# ASSET NAME
|
||||
fields.append({
|
||||
'id': 'ARTESIA.FIELD.ASSET NAME',
|
||||
'value': {'value': {'value': clean_filename}}
|
||||
})
|
||||
if 'ARTESIA.FIELD.ASSET NAME' in fields_by_id:
|
||||
set_string_value(fields_by_id['ARTESIA.FIELD.ASSET NAME'], clean_filename)
|
||||
|
||||
# DESCRIPTION (from subject_title)
|
||||
if parsed_filename.get('subject_title'):
|
||||
fields.append({
|
||||
'id': 'ARTESIA.FIELD.ASSET DESCRIPTION',
|
||||
'value': {'value': {'value': parsed_filename['subject_title']}}
|
||||
})
|
||||
# DESCRIPTION
|
||||
if parsed_filename.get('subject_title') and 'ARTESIA.FIELD.ASSET DESCRIPTION' in fields_by_id:
|
||||
set_string_value(fields_by_id['ARTESIA.FIELD.ASSET DESCRIPTION'], parsed_filename['subject_title'])
|
||||
|
||||
# BRAND (use code for now, could add lookup later)
|
||||
if parsed_filename.get('brand_code'):
|
||||
fields.append({
|
||||
'id': 'FERRERO.FIELD.BRAND',
|
||||
'value': {'value': {'value': parsed_filename['brand_code']}}
|
||||
})
|
||||
# Note: BRAND and COUNTRY are NOT set in the metadata payload.
|
||||
# They are inherited from the DAM folder structure.
|
||||
|
||||
# COUNTRY (map ISO code to DAM code)
|
||||
if parsed_filename.get('country_code'):
|
||||
dam_country_code = self._map_country_code(parsed_filename['country_code'])
|
||||
fields.append({
|
||||
'id': 'FERRERO.FIELD.COUNTRY',
|
||||
'value': {'value': {'value': dam_country_code}}
|
||||
})
|
||||
|
||||
# LANGUAGE (use code for now)
|
||||
if parsed_filename.get('language_code'):
|
||||
fields.append({
|
||||
'id': 'FERRERO.FIELD.LANGUAGES',
|
||||
'value': {'value': {'value': parsed_filename['language_code']}}
|
||||
})
|
||||
|
||||
# ASSET TYPE (use code for now)
|
||||
# ASSET TYPE (use config field ID, map code via lookup)
|
||||
if parsed_filename.get('asset_type'):
|
||||
fields.append({
|
||||
'id': 'FERRERO.FIELD.ASSET TYPE',
|
||||
'value': {'value': {'value': parsed_filename['asset_type']}}
|
||||
})
|
||||
asset_type_field_id = 'FERRERO.FIELD.ASSET TYPE'
|
||||
for field_id, config in self.filename_updates.items():
|
||||
if config.get('source') == 'asset_type':
|
||||
asset_type_field_id = field_id
|
||||
break
|
||||
|
||||
# STATE (force to Local)
|
||||
fields.append({
|
||||
'id': 'FERRERO.FIELD.STATE',
|
||||
'value': {'value': {'value': 'Local'}}
|
||||
})
|
||||
mapped_asset_type = self._map_asset_type(parsed_filename['asset_type'])
|
||||
if asset_type_field_id in fields_by_id:
|
||||
set_domained_value(fields_by_id[asset_type_field_id], mapped_asset_type)
|
||||
|
||||
logger.info("Built {} fields from filename (folder-only mode)".format(len(fields)))
|
||||
# STATE (forced to Local)
|
||||
if 'FERRERO.FIELD.STATE' in fields_by_id:
|
||||
set_domained_value(fields_by_id['FERRERO.FIELD.STATE'], 'Local')
|
||||
|
||||
# VALIDITY DATES (Start = Today, End = Today + 1 Year)
|
||||
try:
|
||||
today = datetime.now()
|
||||
one_year_later = today + timedelta(days=365)
|
||||
start_date_str = today.strftime('%m/%d/%Y')
|
||||
end_date_str = one_year_later.strftime('%m/%d/%Y')
|
||||
|
||||
if 'FERRERO.FIELD.ASSET VALIDITY START PERIOD' in fields_by_id:
|
||||
set_string_value(fields_by_id['FERRERO.FIELD.ASSET VALIDITY START PERIOD'], start_date_str)
|
||||
if 'FERRERO.FIELD.ASSET VALIDITY END PERIOD' in fields_by_id:
|
||||
set_string_value(fields_by_id['FERRERO.FIELD.ASSET VALIDITY END PERIOD'], end_date_str)
|
||||
except Exception as e:
|
||||
logger.error("Failed to set validity dates in folder-only mode: {}".format(str(e)))
|
||||
|
||||
logger.info("Built {} fields from template (folder-only mode)".format(len(fields)))
|
||||
|
||||
return fields
|
||||
|
||||
|
|
@ -757,7 +1166,56 @@ class MetadataExtractorMVP:
|
|||
}
|
||||
})
|
||||
logger.info("Added new Master Asset ID field: {}".format(master_field_id))
|
||||
|
||||
|
||||
return mvp_fields
|
||||
|
||||
def _add_master_asset_ids_field(self, mvp_fields, master_opentext_ids):
|
||||
"""
|
||||
Add FERRERO.MASTERASSETIDS tabular field with multiple master asset IDs
|
||||
Supports Many-to-Many relationship between derivatives and masters
|
||||
|
||||
Args:
|
||||
mvp_fields: List of MVP fields
|
||||
master_opentext_ids: List of DAM Asset IDs of master assets
|
||||
|
||||
Returns:
|
||||
Updated mvp_fields list with FERRERO.MASTERASSETIDS
|
||||
"""
|
||||
if not master_opentext_ids or len(master_opentext_ids) == 0:
|
||||
logger.info("No master_opentext_ids provided - skipping FERRERO.MASTERASSETIDS field")
|
||||
return mvp_fields
|
||||
|
||||
# Check if field already exists
|
||||
for field in mvp_fields:
|
||||
if self._get_field_id(field) == 'FERRERO.MASTERASSETIDS':
|
||||
logger.info("FERRERO.MASTERASSETIDS already present - skipping")
|
||||
return mvp_fields
|
||||
|
||||
# Build values array with all master asset IDs
|
||||
values = []
|
||||
for master_id in master_opentext_ids:
|
||||
values.append({
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': False,
|
||||
'is_locked': False,
|
||||
'value': {
|
||||
'type': 'string',
|
||||
'value': master_id
|
||||
}
|
||||
})
|
||||
|
||||
# Create tabular field
|
||||
new_field = {
|
||||
'id': 'FERRERO.MASTERASSETIDS',
|
||||
'parent_table_id': 'FERRERO.TABULAR.FIELD.MASTERASSETIDS',
|
||||
'type': 'com.artesia.metadata.MetadataTableField',
|
||||
'values': values
|
||||
}
|
||||
|
||||
mvp_fields.append(new_field)
|
||||
logger.info("Added FERRERO.MASTERASSETIDS field with {} master asset ID(s): {}".format(
|
||||
len(values), ', '.join(master_opentext_ids[:3]) + ('...' if len(master_opentext_ids) > 3 else '')))
|
||||
|
||||
return mvp_fields
|
||||
|
||||
def _get_field_id(self, field):
|
||||
|
|
|
|||
|
|
@ -203,8 +203,28 @@ class MetadataExtractorMVP:
|
|||
# Update the field
|
||||
for field in mvp_fields:
|
||||
if field.get('id') == field_id:
|
||||
self._set_field_value(field, value)
|
||||
logger.info("Updated {} from filename: {}".format(field_id, value))
|
||||
# For tabular fields (like MAIN_LANGUAGES), update the 'values' array
|
||||
# The DAM reads from 'values' (plural), not 'value' (singular)
|
||||
if field.get('type') == 'com.artesia.metadata.MetadataTableField' or 'values' in field:
|
||||
field['values'] = [
|
||||
{
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'is_locked': False,
|
||||
'value': {
|
||||
'expired_value': False,
|
||||
'field_value': {
|
||||
'type': 'string',
|
||||
'value': value
|
||||
},
|
||||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
}
|
||||
]
|
||||
logger.info("Updated tabular field {} values array from filename: {}".format(field_id, value))
|
||||
else:
|
||||
self._set_field_value(field, value)
|
||||
logger.info("Updated {} from filename: {}".format(field_id, value))
|
||||
break
|
||||
|
||||
# Apply country code mapping (ISO -> DAM codes)
|
||||
|
|
|
|||
148
Python-Version/test_masterassetids.py
Normal file
148
Python-Version/test_masterassetids.py
Normal file
|
|
@ -0,0 +1,148 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
Test script to verify MASTERASSETIDS field implementation
|
||||
Shows master assets and their potential derivatives
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import psycopg2
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Load env vars from current directory
|
||||
script_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
load_dotenv(os.path.join(script_dir, '.env'))
|
||||
|
||||
try:
|
||||
conn = psycopg2.connect(
|
||||
host=os.getenv('DB_HOST', 'localhost'),
|
||||
port=os.getenv('DB_PORT', '5437'),
|
||||
database='ferrero_tracking',
|
||||
user=os.getenv('DB_USER'),
|
||||
password=os.getenv('DB_PASSWORD')
|
||||
)
|
||||
cursor = conn.cursor()
|
||||
|
||||
print("=" * 80)
|
||||
print("MASTERASSETIDS FIELD TESTING REPORT")
|
||||
print("=" * 80)
|
||||
|
||||
# 1. Show master assets available for testing
|
||||
print("\n📋 MASTER ASSETS (Available for Testing)")
|
||||
print("-" * 80)
|
||||
cursor.execute("""
|
||||
SELECT
|
||||
tracking_id,
|
||||
opentext_id,
|
||||
local_campaign_id,
|
||||
original_filename,
|
||||
created_at
|
||||
FROM master_assets
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 10
|
||||
""")
|
||||
|
||||
print(f"{'Tracking ID':<12} {'OpenText ID':<45} {'Campaign':<15} {'Filename':<30}")
|
||||
print("-" * 80)
|
||||
for row in cursor.fetchall():
|
||||
tracking_id, opentext_id, campaign_id, filename, created_at = row
|
||||
filename_short = (filename[:27] + '...') if filename and len(filename) > 30 else filename or 'N/A'
|
||||
print(f"{tracking_id:<12} {opentext_id:<45} {campaign_id:<15} {filename_short:<30}")
|
||||
|
||||
# 2. Show derivative assets (if any exist)
|
||||
print("\n\n📦 DERIVATIVE ASSETS (Uploaded from Agency)")
|
||||
print("-" * 80)
|
||||
cursor.execute("""
|
||||
SELECT
|
||||
da.tracking_id,
|
||||
da.dam_asset_id,
|
||||
da.derivative_filename,
|
||||
ma.opentext_id as master_opentext_id,
|
||||
ma.local_campaign_id,
|
||||
da.created_at
|
||||
FROM derivative_assets da
|
||||
LEFT JOIN master_assets ma ON da.tracking_id = ma.tracking_id
|
||||
ORDER BY da.created_at DESC
|
||||
LIMIT 10
|
||||
""")
|
||||
|
||||
derivative_rows = cursor.fetchall()
|
||||
if derivative_rows:
|
||||
print(f"{'Tracking ID':<12} {'Derivative DAM ID':<45} {'Master DAM ID (should be in MASTERASSETIDS)':<50}")
|
||||
print("-" * 80)
|
||||
for row in derivative_rows:
|
||||
tracking_id, dam_asset_id, filename, master_opentext_id, campaign_id, created_at = row
|
||||
print(f"{tracking_id:<12} {dam_asset_id or 'N/A':<45} {master_opentext_id or 'N/A':<50}")
|
||||
else:
|
||||
print("(No derivative assets found)")
|
||||
print("\nℹ️ Derivatives are created when Agency returns localized assets (A2→A3 flow)")
|
||||
|
||||
# 3. Show campaigns ready for testing
|
||||
print("\n\n🧪 CAMPAIGNS READY FOR TESTING")
|
||||
print("-" * 80)
|
||||
cursor.execute("""
|
||||
SELECT
|
||||
cs.campaign_number,
|
||||
cs.campaign_name,
|
||||
cs.status,
|
||||
COUNT(ma.id) as master_count,
|
||||
MAX(cs.updated_at) as last_updated
|
||||
FROM campaign_status cs
|
||||
LEFT JOIN master_assets ma ON cs.campaign_number = ma.local_campaign_id
|
||||
WHERE cs.status IN ('A2', 'A3')
|
||||
GROUP BY cs.campaign_number, cs.campaign_name, cs.status
|
||||
ORDER BY last_updated DESC
|
||||
""")
|
||||
|
||||
test_campaigns = cursor.fetchall()
|
||||
if test_campaigns:
|
||||
print(f"{'Campaign':<15} {'Status':<8} {'Master Assets':<15} {'Campaign Name':<40}")
|
||||
print("-" * 80)
|
||||
for row in test_campaigns:
|
||||
campaign_num, campaign_name, status, count, last_updated = row
|
||||
print(f"{campaign_num:<15} {status:<8} {count:<15} {campaign_name[:37]}")
|
||||
else:
|
||||
print("(No campaigns in A2 or A3 status)")
|
||||
|
||||
# 4. Get a sample tracking ID for testing
|
||||
print("\n\n🔬 TEST SCENARIO")
|
||||
print("-" * 80)
|
||||
cursor.execute("""
|
||||
SELECT tracking_id, opentext_id, local_campaign_id, original_filename
|
||||
FROM master_assets
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 1
|
||||
""")
|
||||
|
||||
sample = cursor.fetchone()
|
||||
if sample:
|
||||
tracking_id, opentext_id, campaign_id, filename = sample
|
||||
print(f"Sample Master Asset for Testing:")
|
||||
print(f" Tracking ID: {tracking_id}")
|
||||
print(f" OpenText ID: {opentext_id}")
|
||||
print(f" Campaign: {campaign_id}")
|
||||
print(f" Filename: {filename or 'N/A'}")
|
||||
print(f"\nTo test MASTERASSETIDS field:")
|
||||
print(f" 1. Upload a derivative file to Box with tracking ID: {tracking_id}")
|
||||
print(f" 2. Run: python scripts/a2_to_a3_upload_polling.py --dryrun")
|
||||
print(f" 3. Check for FERRERO.MASTERASSETIDS field with value: {opentext_id}")
|
||||
print(f"\nNote: Field is only active in PPR environment (ppr.dam.ferrero.com)")
|
||||
|
||||
# 5. Environment check
|
||||
print("\n\n🌍 ENVIRONMENT CONFIGURATION")
|
||||
print("-" * 80)
|
||||
dam_url = os.getenv('DAM_BASE_URL', 'Not configured')
|
||||
print(f"DAM Base URL: {dam_url}")
|
||||
if 'ppr.dam.ferrero.com' in dam_url:
|
||||
print("Environment: PPR (MASTERASSETIDS field is ENABLED ✅)")
|
||||
elif 'dam.ferrero.com' in dam_url:
|
||||
print("Environment: PROD (MASTERASSETIDS field is DISABLED ⚠️)")
|
||||
else:
|
||||
print("Environment: Unknown")
|
||||
|
||||
print("\n" + "=" * 80)
|
||||
|
||||
conn.close()
|
||||
|
||||
except Exception as e:
|
||||
print(f"❌ Error: {e}")
|
||||
sys.exit(1)
|
||||
94
Python-Version/test_multiple_master_ids.py
Normal file
94
Python-Version/test_multiple_master_ids.py
Normal file
|
|
@ -0,0 +1,94 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
Test script to demonstrate MASTERASSETIDS field with multiple master asset IDs
|
||||
This creates a test JSON structure showing how multiple master assets would be linked
|
||||
"""
|
||||
import json
|
||||
|
||||
# Example: A localized asset (derivative) that references TWO master assets
|
||||
# Master 1: fc5c389776516bb58044c7d4bf479da458599baf (tracking: BqB8vo)
|
||||
# Master 2: ad3948d72ea8550a338a600ae87a1bdd1968b066 (tracking: SfUQ7m)
|
||||
|
||||
test_field_structure = {
|
||||
'id': 'FERRERO.MASTERASSETIDS',
|
||||
'parent_table_id': 'FERRERO.TABULAR.FIELD.MASTERASSETIDS',
|
||||
'type': 'com.artesia.metadata.MetadataTableField',
|
||||
'values': [
|
||||
# First master asset ID
|
||||
{
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'is_locked': False,
|
||||
'value': {
|
||||
'field_value': {
|
||||
'type': 'string',
|
||||
'value': 'fc5c389776516bb58044c7d4bf479da458599baf'
|
||||
},
|
||||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
},
|
||||
# Second master asset ID
|
||||
{
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'is_locked': False,
|
||||
'value': {
|
||||
'field_value': {
|
||||
'type': 'string',
|
||||
'value': 'ad3948d72ea8550a338a600ae87a1bdd1968b066'
|
||||
},
|
||||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
},
|
||||
# Third master asset ID (optional)
|
||||
{
|
||||
'cascading_domain_value': False,
|
||||
'domain_value': True,
|
||||
'is_locked': False,
|
||||
'value': {
|
||||
'field_value': {
|
||||
'type': 'string',
|
||||
'value': '020d76f957ec9f4ec0b18035a2d012cd3fd376c2'
|
||||
},
|
||||
'type': 'com.artesia.metadata.DomainValue'
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
print("=" * 80)
|
||||
print("MULTIPLE MASTER ASSET IDS - TEST STRUCTURE")
|
||||
print("=" * 80)
|
||||
print()
|
||||
print("Field ID:", test_field_structure['id'])
|
||||
print("Parent Table:", test_field_structure['parent_table_id'])
|
||||
print("Number of Master Asset IDs:", len(test_field_structure['values']))
|
||||
print()
|
||||
print("Master Asset IDs:")
|
||||
for i, value_obj in enumerate(test_field_structure['values'], 1):
|
||||
master_id = value_obj['value']['field_value']['value']
|
||||
print(f" {i}. {master_id}")
|
||||
print()
|
||||
print("Full JSON Structure:")
|
||||
print("-" * 80)
|
||||
print(json.dumps(test_field_structure, indent=2))
|
||||
print()
|
||||
print("=" * 80)
|
||||
print("TESTING NOTES")
|
||||
print("=" * 80)
|
||||
print()
|
||||
print("To test if DAM accepts multiple IDs:")
|
||||
print("1. Check if FERRERO.TABULAR.FIELD.MASTERASSETIDS schema allows multiple rows")
|
||||
print("2. Verify with DAM admin if field has 'Allow Multiple Values' enabled")
|
||||
print("3. Test upload with this structure to PPR environment")
|
||||
print()
|
||||
print("Current Implementation:")
|
||||
print(" - Code adds ONE master ID (from tracking ID lookup)")
|
||||
print(" - Supports Many-to-Many relationship conceptually")
|
||||
print(" - Array structure ready for multiple values")
|
||||
print()
|
||||
print("To enable multiple IDs in production:")
|
||||
print(" - Agency tool needs to send list of master tracking IDs")
|
||||
print(" - Database schema needs multiple master references")
|
||||
print(" - Code modification needed to look up multiple masters")
|
||||
print()
|
||||
Loading…
Add table
Reference in a new issue