Docs: Refresh A1 empty-folder doc and LTD asset type notes
A1_RETRY_LOGIC.md updated to reflect the 2026-04-28 rework: empty folders are now treated as expected workflow (silent skip + one-time warning at poll 20, no auto permanent-fail), while the original 3-strikes-then-permanently-fail behavior is preserved for genuine folder errors via the mark_failed_at_max flag. README.md adds LTD (Licensing Translation Document) to the asset type override section alongside EOL, and notes that empty overrides remove fields while non-empty overrides on non-MVP fields are appended. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
nickviljoen
parent
ba4f1a9bf7
commit
28586308d7
2 changed files with 57 additions and 50 deletions
|
|
@ -1,83 +1,86 @@
|
|||
# A1→A2 Empty Folder Retry Logic
|
||||
## Preventing Infinite Error Emails for Empty Campaigns
|
||||
# A1→A2 Empty Folder Handling
|
||||
|
||||
**Purpose:** Avoid sending "no assets found" error emails every 3 minutes indefinitely when a campaign is set to A1 but has no master assets.
|
||||
**Purpose:** Avoid spam emails and false-positive permanent failures for the common workflow where campaign managers create an A1 campaign before uploading the master assets.
|
||||
|
||||
**Author:** Claude Code
|
||||
**Date:** January 31, 2026
|
||||
**Related Files:**
|
||||
**Initial implementation:** January 31, 2026
|
||||
**Reworked:** April 28, 2026 — empty folders are now treated as expected client workflow rather than failures.
|
||||
|
||||
**Related files:**
|
||||
- `scripts/a1_to_a2_box_uploader.py` (main script)
|
||||
- `scripts/shared/database.py` (retry tracking methods)
|
||||
- `database/migrations/003_add_a1_retry_tracking.sql` (schema)
|
||||
|
||||
---
|
||||
|
||||
## How It Works
|
||||
## How It Works (current behavior)
|
||||
|
||||
### The Problem
|
||||
Previously, when a campaign had status A1 but the Master Assets folder was empty:
|
||||
- System sent error email every 3 minutes
|
||||
- Campaign remained in A1 status forever
|
||||
- No distinction between temporary and permanent failures
|
||||
- Notification fatigue for support team
|
||||
|
||||
### The Solution
|
||||
Three-attempt retry mechanism with permanent failure tracking:
|
||||
### The empty-folder case (most common)
|
||||
When a campaign is at A1 in DAM but the Master Assets folder is empty, the script treats this as a normal pre-asset state, not a failure.
|
||||
|
||||
**Flow:**
|
||||
1. Campaign in A1 status with no assets → Attempt 1 (email sent, retry_count=1)
|
||||
2. Still no assets after 3 minutes → Attempt 2 (email sent, retry_count=2)
|
||||
3. Still no assets after 6 minutes → Attempt 3 (email sent, retry_count=3, permanently_failed=TRUE)
|
||||
4. Campaign now skipped on all future runs → Manual reset required
|
||||
1. Every poll: `a1_retry_count` is incremented for visibility, the script logs `No master assets yet (poll N) - skipping until assets appear`, and exits silently.
|
||||
2. At poll 20 (~1 hour at the 3-minute orchestrator cadence) the script sends a single `a1_to_a2_no_assets_warning` email so genuinely-stuck campaigns still surface.
|
||||
3. After poll 20, the script keeps skipping silently. **`a1_permanently_failed` is never auto-set for empty folders.**
|
||||
4. When assets eventually appear and A1→A2 succeeds, `db.reset_a1_retry()` clears the counter automatically.
|
||||
|
||||
### Database Tracking
|
||||
The threshold lives in `scripts/a1_to_a2_box_uploader.py` as `EMPTY_FOLDER_WARNING_THRESHOLD = 20`.
|
||||
|
||||
Four new fields in `campaign_status` table:
|
||||
- `a1_retry_count` (INTEGER): Number of failed attempts (0-3)
|
||||
### The genuine-error case
|
||||
The 3-retries-then-permanently-fail behavior **still exists** for actual folder-level errors (e.g. `Assets folder not found (tried Master Assets)`), which are caught by the script's exception handler. These DO mark `a1_permanently_failed=TRUE` after 3 failures and DO send the retry / permanently-failed emails.
|
||||
|
||||
`db.increment_a1_retry()` accepts `mark_failed_at_max=True|False` to switch between the two behaviors. The empty-folder branch passes `False`; the exception handler passes `True` (default).
|
||||
|
||||
### Queue-slot filter
|
||||
The A1→A2 script processes up to 2 campaigns per run (`campaigns[:2]`). Permanently-failed campaigns are filtered out **before** the slot cap so they no longer block the queue (`scripts/a1_to_a2_box_uploader.py:652`).
|
||||
|
||||
### Database tracking
|
||||
|
||||
Four fields on the `campaign_status` table:
|
||||
- `a1_retry_count` (INTEGER): Number of polls where the folder was empty / errored. For empty-folder cases this can grow unbounded; reset on success.
|
||||
- `a1_last_retry_at` (TIMESTAMP): When last attempt occurred
|
||||
- `a1_permanently_failed` (BOOLEAN): TRUE after 3 failures
|
||||
- `a1_failure_reason` (TEXT): Why it failed (e.g., "No master assets found")
|
||||
- `a1_permanently_failed` (BOOLEAN): TRUE only via the genuine-error path (after 3 failures), never via the empty-folder path
|
||||
- `a1_failure_reason` (TEXT): Why it failed (e.g., "Assets folder not found (tried Master Assets)")
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Maximum Retry Attempts
|
||||
|
||||
**Current Setting:** 3 attempts
|
||||
|
||||
**To Change:** Edit `/Users/nickviljoen/Desktop/Ferrero/ferrero-opentext/Python-Version/scripts/shared/database.py`
|
||||
|
||||
### Empty-folder warning threshold
|
||||
`scripts/a1_to_a2_box_uploader.py`:
|
||||
```python
|
||||
def increment_a1_retry(self, campaign_id, campaign_number, campaign_name, reason):
|
||||
"""..."""
|
||||
# Maximum retry attempts before marking as permanently failed
|
||||
MAX_RETRIES = 3 # CHANGE THIS NUMBER
|
||||
EMPTY_FOLDER_WARNING_THRESHOLD = 20 # ~1 hour at 3-min poll cadence
|
||||
```
|
||||
Send the one-time warning sooner/later by adjusting this constant.
|
||||
|
||||
**Recommendation:** Keep at 3. This allows:
|
||||
- Immediate notification (attempt 1)
|
||||
- Short-term retry (attempt 2 after 3 min)
|
||||
- Medium-term retry (attempt 3 after 6 min)
|
||||
- Permanent failure (after 9 minutes total)
|
||||
### Genuine-error retry attempts
|
||||
`scripts/shared/database.py` → `increment_a1_retry()`:
|
||||
```python
|
||||
MAX_RETRIES = 3
|
||||
```
|
||||
Applies only when the caller passes `mark_failed_at_max=True` (default), i.e. the exception handler in `process_campaign()`. The empty-folder branch passes `False` and is unaffected.
|
||||
|
||||
---
|
||||
|
||||
## Email Notifications
|
||||
|
||||
### Retry Email (Attempts 1-2)
|
||||
### Empty-folder warning (one-time, at poll 20)
|
||||
**Template:** `a1_to_a2_no_assets_warning`
|
||||
**Subject:** ⚠️ Campaign in A1 with no assets yet - {campaign_name}
|
||||
**Recipients:** Error notification list
|
||||
**Sent:** exactly once per stuck campaign, when `a1_retry_count == 20`. Counter resets on success, so a future re-stuck event would warn again.
|
||||
|
||||
### Genuine-error retry email (attempts 1–2)
|
||||
**Template:** `a1_to_a2_no_assets_retry`
|
||||
**Subject:** ⚠️ No Assets Found (Attempt X/3) - Campaign {name}
|
||||
**Recipients:** Error notification list
|
||||
**Content:**
|
||||
- Current retry count
|
||||
- Remaining attempts
|
||||
- What happens next
|
||||
**Trigger:** non-empty-folder errors caught by `process_campaign()`'s exception handler.
|
||||
|
||||
### Final Failure Email (Attempt 3)
|
||||
### Genuine-error final failure (attempt 3)
|
||||
**Template:** `a1_to_a2_permanently_failed`
|
||||
**Subject:** ❌ PERMANENTLY FAILED - Campaign {name} (No Assets After 3 Attempts)
|
||||
**Recipients:** Error notification list
|
||||
**Content:**
|
||||
- Campaign marked as permanently failed
|
||||
- Campaign marked as permanently failed (campaign filtered from future queue runs)
|
||||
- Required actions to fix
|
||||
- SQL command to manually reset
|
||||
|
||||
|
|
|
|||
|
|
@ -974,9 +974,11 @@ Each file defines: MVP fields, filename update rules, forced values, defaults, a
|
|||
|
||||
`config/asset_representation_template.json` is the reference template for folder-only mode (`-N` flag uploads). It contains the full field metadata structure that the DAM API requires for asset creation. This template was provided by the client and should be updated if the DAM metadata model changes.
|
||||
|
||||
### Asset Type Overrides (EOL Example)
|
||||
### Asset Type Overrides (EOL / LTD)
|
||||
|
||||
Certain asset types trigger field overrides configured in the field mappings file. For example, **EOL (External Legal Opinion)** overrides:
|
||||
Certain asset types trigger field overrides configured in the field mappings file. Currently configured for both PPR and PROD:
|
||||
|
||||
**EOL (External Legal Opinion)**
|
||||
- Agency Name = "-"
|
||||
- Production House = "-"
|
||||
- Main Languages = "Global"
|
||||
|
|
@ -984,7 +986,9 @@ Certain asset types trigger field overrides configured in the field mappings fil
|
|||
- Licensing = "No"
|
||||
- Validity dates removed
|
||||
|
||||
These overrides are applied after all other field processing and take final precedence.
|
||||
**LTD (Licensing Translation Document)** — supports the EOL workflow with translated license claims. Same overrides as EOL, plus a fixed Description: `"Translation of License claim - For approval purposes only"`. Currently mapped to the same DAM-side code (`externallegalopinion`) as a placeholder pending client confirmation.
|
||||
|
||||
These overrides are applied after all other field processing and take final precedence. An empty-string override removes the field; a non-empty override targeting a field that isn't in `mvp_fields` will be appended as a simple string field.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue