Oliver/ferrero-opentext
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
157 commits 2 branches 0 tags 82 MiB
Commit graph

3 commits

Author SHA1 Message Date
DJP
d818a8b6a8 Update comprehensive README and reorganize documentation files 
Major README overhaul with complete deployment and configuration guide.
Moved old docs to tests/ folder for archive.

README.md UPDATES (880 lines - completely rewritten):
✓ Table of contents with navigation
✓ Complete overview of all 4 workflows + daily report
✓ Detailed authentication section (OAuth2 vs mTLS)
✓ Box-config.json location explanation
✓ Server deployment step-by-step guide
✓ Database setup (Docker + native PostgreSQL)
✓ Cron job examples for all workflows
✓ Comprehensive troubleshooting section
✓ Security checklist
✓ Monitoring and log rotation details
✓ Common SQL queries
✓ File structure diagram

KEY SECTIONS ADDED:
1. What's Included - All 5 scripts explained
2. Quick Start - Local setup guide
3. Server Deployment - 6-step process with commands
4. Workflows - Detailed process for each (A1→A2, A5→A6, B1→B2, A2→A3, Daily Report)
5. Authentication - OAuth2 vs mTLS with examples
6. Configuration - All .env variables documented
7. Database - Schema, setup, queries
8. Monitoring - Logs, emails, database queries
9. Troubleshooting - Common issues + solutions
10. File Structure - Complete directory tree

BOX-CONFIG.JSON LOCATION DOCUMENTED:
✓ Must be one folder up from Python-Version
✓ Referenced as ../Box-config.json in config.yaml
✓ Server deployment instructions include copying both files
✓ Troubleshooting section explains file not found errors

MTLS DOCUMENTATION:
✓ Different base URL explained (dev-auth.app-api.ferrero.com)
✓ --auth-pfx flag usage
✓ Whitelisted IP requirement noted
✓ Certificate testing commands

REORGANIZATION:
- Moved old DEPLOYMENT.md → tests/DEPLOYMENT.md (archive)
- Moved old WORKFLOW_DIAGRAMS.md → tests/WORKFLOW_DIAGRAMS.md (archive)
- New DEPLOYMENT_GUIDE.md is the current deployment doc
- README.md is now comprehensive one-stop documentation

Changes:
- Python-Version/README.md (completely rewritten, 880 lines)
- Moved 2 old docs to tests/ folder
- Added test files to tests/ folder

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-05 10:57:11 -05:00
DJP
2943277047 Add comprehensive DEPLOYMENT.md and update README for production server 
Created DEPLOYMENT.md:
✅ Complete step-by-step production server deployment guide
✅ Python 3.6 server requirements and setup
✅ Virtual environment creation
✅ Credential configuration
✅ Connection testing procedures
✅ Cron job setup (A1→A2 every 5 minutes)
✅ Webhook server setup (A2→A3)
✅ Process monitoring scripts
✅ Security best practices (file permissions, .env protection)
✅ Troubleshooting guide (all common issues)
✅ Debugging procedures
✅ Health check scripts
✅ Log monitoring
✅ Configuration update procedures (add fields, change recipients, etc.)
✅ Emergency procedures (stop/start/restart)

Updated README.md:
✅ Added references to DEPLOYMENT.md
✅ Updated with correct Box folder IDs
✅ Production-ready status
✅ Clear documentation hierarchy
✅ Make.com webhook integration noted
✅ Email configuration documented

Key Documentation:
- DEPLOYMENT.md: Production server deployment (complete guide)
- README.md: Quick reference and local testing
- PYTHON_AUTOMATION_PLAN.md: Architecture and design

All guides updated with:
- Correct Box folders (348304357505 for A1→A2, 348526703108 for A2→A3)
- Folder naming: C000000078-Campaign_Name
- Make.com webhook URL
- SMTP/Mailgun email configuration
- Single-run mode (process one campaign and exit)
- All-done checks before status updates

Ready for production deployment on Python 3.6 server!

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-10-30 18:59:12 -04:00
DJP
b4e004c822 Complete Python automation implementation - All components built 
MAJOR MILESTONE: Complete Python automation system created!

Components Implemented:
✅ Box Client (box_client.py)
   - JWT authentication via boxsdk
   - Upload with tracking ID suffix
   - Download files
   - Campaign folder creation
   - Connection testing

✅ Database Client (database.py)
   - PostgreSQL connection pooling
   - generate_unique_tracking_id()
   - store_master_asset() with full_metadata JSONB
   - get_master_asset(tracking_id)
   - check_campaign_upload_complete() - ALL-DONE CHECK!
   - store_derivative_asset()
   - Connection testing

✅ Filename Parser (filename_parser.py)
   - V2 naming convention parser (ported from PHP)
   - parse_filename() - 10 components
   - strip_upload_components() - Remove Job# and Tracking ID
   - Strict validation with detailed errors

✅ Metadata Extractor MVP (metadata_extractor_mvp.py)
   - Extract 28 MVP fields from master
   - Update fields from V2 filename (Description, Language, State)
   - Add missing fields with defaults
   - Build asset representation for upload

✅ Notifier (notifier.py)
   - Mailgun email integration
   - Outgoing webhook sender
   - Email templates (success, error, partial, critical)
   - Configurable recipients

Main Scripts:
✅ A1→A2 Download (a1_to_a2_download.py)
   - Poll DAM every 5 minutes for A1 campaigns
   - Download all master assets
   - Upload to Box with tracking IDs
   - Store in DB with full metadata
   - ALL-DONE CHECK before status update
   - Update A1→A2 only if all assets successful
   - Send webhook with campaign ID/number
   - Email notifications

✅ A2→A3 Upload (a2_to_a3_upload.py)
   - Flask webhook receiver for Box uploads
   - Signature validation
   - Async task queue processing
   - Parse V2 filenames
   - Load master metadata
   - Extract MVP fields
   - Upload to DAM
   - ALL-DONE CHECK for campaign
   - Update A2→A3 when all assets uploaded
   - Send webhook notifications

✅ Test Connection Script (test_connection.py)
   - Verify DAM, Box, Database connectivity
   - Quick health check

✅ README.md
   - Complete setup guide
   - Usage instructions
   - Configuration examples
   - Troubleshooting

Key Features:
- Python 3.6+ compatible (server requirement)
- Virtual environment isolated
- Configuration-driven (YAML files)
- Easy field updates (no code changes)
- Environment switching (staging/production)
- Comprehensive error handling
- Email + webhook notifications
- Retry logic
- All-done checks before status updates
- Campaign webhook notifications

Ready for testing locally with Python 3.10!

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-10-30 16:49:14 -04:00