ferrero-opentext/Python-Version/RUN_ORCHESTRATOR.md

Running the Orchestrator

Quick Start - Watch in Real Time

To run the orchestrator and watch it in real-time (daemon mode):

cd /Users/daveporter/Desktop/CODING-2024/Ferrero-Opentext/Python-Version
python3 scripts/orchestrator.py --daemon

This will:

• Run continuously in the foreground
• Check every minute for tasks that need to run
• Show all output in your terminal
• Press Ctrl+C to stop

---

Other Run Modes

Force Run All Tasks Immediately

python3 scripts/orchestrator.py --force

Runs all tasks right now, ignoring their schedules.

Single Check (Cron Mode)

python3 scripts/orchestrator.py

Checks once, runs any due tasks, then exits. This is what cron would call.

---

Current Task Schedule

• A1→A2 Box Uploader: Every 5 minutes (processes 2 campaigns)
• A2→A3 Upload Polling: Every 5 minutes (processes ALL files)
• A4 Box Uploader: Every 10 minutes
• A4 Webhook Monitor: Every 5 minutes
• A5→A6 Download: Every 5 minutes
• B1→B2 Download: Every 10 minutes
• Daily Report: Once per day at 7 PM

---

Off-Hours Configuration

Overview

The orchestrator automatically reduces task frequency during off-hours to minimize system load during low-activity periods.

What changes during off-hours:

• All tasks run less frequently (only at 0 and 30 minute marks)
• Example: A 3-minute task normally runs at minutes 0, 3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33, 36, etc.
• During off-hours: Runs only at minutes 0 and 30 (every 30 minutes)
• Daily Report (7 PM) remains unchanged

Off-hours definition:

• Late night: 10 PM (22:00) to 5 AM (05:00) every day
• All day Saturday (00:00-23:59)
• All day Sunday (00:00-23:59)

Configuration

Location: scripts/orchestrator-prod.py lines ~88-107

OFF_HOURS_CONFIG = {
    'enabled': True,         # Set to False to disable
    'extra_minutes': 30,     # Minutes to add during off-hours

    'late_night_start': 22,  # Start hour (22 = 10 PM)
    'late_night_end': 5,     # End hour (5 = 5 AM)

    'weekend_days': [5, 6],  # Saturday=5, Sunday=6

    'exempt_tasks': [
        'Daily Report'       # Tasks that ignore off-hours
    ]
}

Examples

Business Hours (Monday 2 PM):

A1→A2: Runs every 3 minutes (0, 3, 6, 9, 12, 15, 18, 21, 24, 27, 30, ...)
A4 Box: Runs every 10 minutes (0, 10, 20, 30, 40, 50)

Off-Hours (Monday 11 PM or Saturday):

A1→A2: Runs every 30 minutes (0, 30)
A4 Box: Runs every 30 minutes (0, 30)
All tasks: Only run at minutes 0 and 30

Customization

Change off-hours timing

Edit orchestrator-prod.py:

# Late night only from midnight to 6 AM
'late_night_start': 0,
'late_night_end': 6,

# Include only Sunday as weekend
'weekend_days': [6],  # 6 = Sunday

Disable off-hours completely

OFF_HOURS_CONFIG = {
    'enabled': False,  # Turns off all off-hours logic
    # ... rest unchanged
}

Exempt specific tasks

'exempt_tasks': [
    'Daily Report',
    'A4 Webhook Monitor'  # This task will run at normal cadence even in off-hours
]

Monitoring

Check orchestrator logs to see current mode:

# Watch for mode changes
tail -f logs/orchestrator.log | grep "MODE"

# Output examples:
# Orchestrator tick: 2026-01-31 14:00:00 [NORMAL MODE]
# Orchestrator tick: 2026-01-31 22:00:00 [OFF-HOURS MODE]
# Adding 30 minutes to all task intervals

Testing

# Test without affecting production
python scripts/orchestrator-prod.py --force

# Look for these log messages:
# [OFF-HOURS MODE] or [NORMAL MODE]
# "Adding 30 minutes to all task intervals"
# "Task 'A1->A2' due (off-hours: 3min + 30min cadence)"

---

Logs

• Orchestrator logs: logs/orchestrator.log
• Individual script logs: logs/a1_to_a2_box.log, logs/a2_to_a3.log, etc.

---

Troubleshooting

If you see git output instead of orchestrator output, make sure you're running:

python3 scripts/orchestrator.py --daemon

NOT:

git pull