Commit graph

6 commits

Author SHA1 Message Date
DJP
20a187a61d feat: Add Python project dependencies. 2025-11-20 22:45:04 -05:00
DJP
d818a8b6a8 Update comprehensive README and reorganize documentation files
Major README overhaul with complete deployment and configuration guide.
Moved old docs to tests/ folder for archive.

README.md UPDATES (880 lines - completely rewritten):
✓ Table of contents with navigation
✓ Complete overview of all 4 workflows + daily report
✓ Detailed authentication section (OAuth2 vs mTLS)
✓ Box-config.json location explanation
✓ Server deployment step-by-step guide
✓ Database setup (Docker + native PostgreSQL)
✓ Cron job examples for all workflows
✓ Comprehensive troubleshooting section
✓ Security checklist
✓ Monitoring and log rotation details
✓ Common SQL queries
✓ File structure diagram

KEY SECTIONS ADDED:
1. What's Included - All 5 scripts explained
2. Quick Start - Local setup guide
3. Server Deployment - 6-step process with commands
4. Workflows - Detailed process for each (A1→A2, A5→A6, B1→B2, A2→A3, Daily Report)
5. Authentication - OAuth2 vs mTLS with examples
6. Configuration - All .env variables documented
7. Database - Schema, setup, queries
8. Monitoring - Logs, emails, database queries
9. Troubleshooting - Common issues + solutions
10. File Structure - Complete directory tree

BOX-CONFIG.JSON LOCATION DOCUMENTED:
✓ Must be one folder up from Python-Version
✓ Referenced as ../Box-config.json in config.yaml
✓ Server deployment instructions include copying both files
✓ Troubleshooting section explains file not found errors

MTLS DOCUMENTATION:
✓ Different base URL explained (dev-auth.app-api.ferrero.com)
✓ --auth-pfx flag usage
✓ Whitelisted IP requirement noted
✓ Certificate testing commands

REORGANIZATION:
- Moved old DEPLOYMENT.md → tests/DEPLOYMENT.md (archive)
- Moved old WORKFLOW_DIAGRAMS.md → tests/WORKFLOW_DIAGRAMS.md (archive)
- New DEPLOYMENT_GUIDE.md is the current deployment doc
- README.md is now comprehensive one-stop documentation

Changes:
- Python-Version/README.md (completely rewritten, 880 lines)
- Moved 2 old docs to tests/ folder
- Added test files to tests/ folder

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-05 10:57:11 -05:00
DJP
1d1284ed5c Add HANDOFF_NEXT_SESSION.md - Complete session handoff document
Created comprehensive handoff document for next session:

Current Status:
 81 commits completed
 88% context used (878k/1000k tokens)
 3 of 4 Python scripts complete
 All systems tested and working

Remaining Work:
 Create a5_to_a6_download.py (Rework workflow)
 Add email templates for A5→A6
 Test A5→A6 script
 Add to cron

Handoff Document Includes:
 What was completed this session
 Detailed A5→A6 implementation guide
 Step-by-step instructions
 Template script to copy (a1_to_a2_download.py)
 All changes needed (search/replace patterns)
 Email template examples
 Testing procedures
 Cron setup
 Database connection info
 Box folder configuration
 Important credentials note
 Verification commands

Estimated Time: 15-20 minutes to complete A5→A6

Next Session:
1. Load HANDOFF_NEXT_SESSION.md
2. Load PROJECT_STATUS_2025-11-03.md
3. Create a5_to_a6_download.py
4. Test and deploy

Session complete - ready for handoff!

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-04 12:30:36 -05:00
DJP
92ad76faae Add comprehensive database schema documentation - DATABASE_SCHEMA.md
Created complete database reference guide:

Schema Documentation:
 Full table definitions (master_assets, derivative_assets)
 All column descriptions with types and purposes
 Index definitions and purposes
 Foreign key constraints
 Trigger definitions

Migration History:
 Session 1: Initial setup
 Session 2: Add full_metadata JSONB (Oct 30)
 Session 3: Add global_master_campaign_id and global_master_folder_id (Nov 3)
 Complete migration SQL for each change
 Before/after comparison
 Impact assessment

Column Reference Table:
 All 30+ columns documented
 Data types and nullability
 Default values
 Purpose and usage notes
 Highlights new columns with 

Query Examples:
 Common queries (recent assets, by tracking ID, etc.)
 Global Master relationship queries
 Campaign completion checks
 Statistics and reporting queries
 Dashboard queries

Operations:
 Backup and restore procedures
 Maintenance tasks (VACUUM, size checks)
 Performance optimization
 Security and permissions
 Troubleshooting guide

Key Features Documented:
- full_metadata JSONB (no truncation!)
- global_master_campaign_id (campaign relationships)
- global_master_folder_id (folder tracking)
- Tracking ID system
- Connection pooling
- Index strategy

Ready for production deployment with complete DB documentation!

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-04 10:24:44 -05:00
DJP
8b576bb598 Add A2→A3 polling version and fix database to use existing columns
Created a2_to_a3_upload_polling.py:
- Polls Box folder (348526703108) instead of webhook
- Works locally (no need for public URL)
- Single-run mode (process one file and exit)
- Can be run via cron every 5 minutes

Why Polling Instead of Webhook:
- Webhooks require public URL (doesn't work on localhost)
- Polling works everywhere (local and server)
- Same functionality, different trigger mechanism

Database Fix:
- Don't create new columns (dam_asset_id, upload_status)
- Use existing schema: tracking_id, derivative_filename, file_extension, status
- Simplified store_derivative_asset() to use existing columns only
- Database now compatible with existing schema

Test Results - A2→A3 Polling:
 Polls Box folder 348526703108
 Finds V2 files with tracking IDs
 Downloads from Box
 Loads master metadata from PostgreSQL
 Builds 27 MVP fields
 Updates Description, State, Language from filename
 Uploads to DAM successfully (Asset ID: 214924)
 Stores derivative record
 Processes one file and exits

Both Scripts Working:
 A1→A2: Downloads from DAM → Box (folder 348304357505)
 A2→A3: Uploads from Box → DAM (folder 348526703108)

Cron Setup:
*/5 * * * * python scripts/a1_to_a2_download.py
*/5 * * * * python scripts/a2_to_a3_upload_polling.py

Complete automation ready for production!

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 19:21:13 -04:00
DJP
363560c06f Fix boxsdk version and add .env file
Changes:
- Downgraded boxsdk to 3.x (compatible API)
- Created .env file with all credentials
- Fixed requirements.txt versions

Python automation now ready for testing:
 Virtual environment created
 All dependencies installed
 Box connection working
 Database connection working
⚠️ DAM OAuth (same creds as PHP, might be temp server issue)

Next steps:
1. Test DAM connection (may need to retry)
2. Run A1→A2 script
3. Monitor logs

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 17:09:49 -04:00