Implements dual backup strategy with daily SQL dumps and weekly binary backups, complete with restore capabilities and health monitoring. Backup System Components: 1. database/backup.sh: - Daily mode: pg_dump SQL dumps (7-day retention) - Weekly mode: pg_basebackup binary backup (latest only) - Automatic cleanup of old backups - Compression (gzip) for space efficiency - Email notifications on failures - Docker-compatible execution 2. database/restore.sh: - Restore from SQL dump backups - Safety backup before restore - Confirmation prompts - Validation and verification - List available backups 3. database/check_backups.sh: - Health check monitoring - Verifies latest backup age (warns if > 25 hours) - Displays backup counts and sizes - Quiet mode for cron automation - Lists all available backups Documentation: - DATABASE_BACKUP_GUIDE.md: Complete backup/restore guide - Automated cron setup - Manual backup procedures - Restore scenarios - Troubleshooting - Disk space management - backups/README.md: Quick reference - Directory structure - Common commands - Retention policy - Security notes Configuration: - Updated .gitignore to exclude backup files - Backup locations: backups/dumps/, backups/basebackups/ - Logs: logs/backup.log, logs/restore.log - Retention: 7 daily dumps + 1 weekly basebackup Cron Schedule (Production): - Daily: 2:00 AM (pg_dump) - Weekly: Sundays 3:00 AM (pg_basebackup) - Health Check: 8:00 AM daily Features: ✅ Automated daily and weekly backups ✅ Dual strategy (logical + physical) ✅ Space-efficient (7-day retention, ~50 MB total) ✅ Safety backups before restore ✅ Email alerts on failures ✅ Health monitoring ✅ Docker-compatible ✅ Tested locally Testing Performed: - Daily backup created successfully (77K compressed) - Backup file integrity verified (gzip test passed) - Health check shows "Backup system healthy" - Restore --list command working - All scripts executable and functional Disk Usage Estimate: - Daily dumps: 7 × ~2 MB = ~14 MB - Weekly backup: 1 × ~30 MB = ~30 MB - Total: ~50 MB maximum 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| README.md | ||
Database Backups Directory
This directory contains automated PostgreSQL backups for the Ferrero Asset Tracking database.
Directory Structure
backups/
├── dumps/ # Daily SQL dumps (pg_dump) - 7 day retention
├── basebackups/ # Weekly binary backups (pg_basebackup) - latest only
├── safety/ # Pre-restore safety backups
├── manual/ # Manual/ad-hoc backups
└── README.md # This file
Automated Backup Schedule
- Daily: 2:00 AM - SQL dump backup (7 days retained)
- Weekly: Sundays 3:00 AM - Full binary backup (latest only)
- Health Check: 8:00 AM - Verify backups are fresh
Quick Commands
Create Backup
# Daily backup
../database/backup.sh --daily
# Weekly backup
../database/backup.sh --weekly
Check Backup Status
# Health check
../database/check_backups.sh
# List all backups
ls -lh dumps/
ls -lh basebackups/
Restore Database
# List available backups
../database/restore.sh --list
# Restore from backup
../database/restore.sh dumps/ferrero_tracking_2025-11-11_02-00.sql.gz
Backup File Naming
Daily dumps:
ferrero_tracking_YYYY-MM-DD_HH-MM.sql.gz
Example: ferrero_tracking_2025-11-11_02-00.sql.gz
Weekly basebackups:
ferrero_tracking_YYYY-MM-DD_HH-MM.tar.gz
Example: ferrero_tracking_2025-11-10_03-00.tar.gz
Retention Policy
- Daily: Keep last 7 days (auto-cleanup)
- Weekly: Keep latest only (auto-cleanup)
- Safety: Manual cleanup after restore
- Manual: Manual cleanup as needed
Expected Disk Usage
- Daily dumps: ~14 MB (7 files × ~2 MB)
- Weekly backup: ~30 MB (1 file)
- Total: ~50 MB maximum
Security
⚠️ Important: Backup files contain sensitive database data.
Permissions:
chmod 700 backups # Only owner can access
chmod 600 *.sql.gz # Backup files read/write owner only
NOT committed to Git: This directory is in .gitignore and will not be pushed to repository.
Monitoring
Check Latest Backup Age
# Find latest daily backup
ls -lt dumps/*.sql.gz | head -1
# Check age in hours
stat -c %Y dumps/*.sql.gz | head -1 | xargs -I {} echo $(( ($(date +%s) - {}) / 3600 )) hours
Verify Backup Worked
# Check backup log
tail -50 ../logs/backup.log
# Look for success markers
grep "✓" ../logs/backup.log | tail -5
What's Included in Backups
All Tables:
master_assets- Master asset trackingderivative_assets- Localized versionscreativex_scores- CreativeX quality scoresasset_events- Audit trailworkflow_state- Workflow trackingcampaign_status- Campaign processing state
All Data:
- Complete JSONB metadata
- All tracking IDs and relationships
- Campaign references
- Full CreativeX extraction data
- Timestamps and audit information
Database Schema:
- Table structures
- Indexes
- Functions
- Triggers
- Constraints
Restore Examples
Restore Yesterday's Backup
# Find yesterday's backup
ls -lh dumps/ | grep $(date -d "yesterday" +%Y-%m-%d)
# Restore it
../database/restore.sh dumps/ferrero_tracking_2025-11-10_02-00.sql.gz
Restore After Accidental Data Deletion
# Immediately create emergency backup of current state
docker exec ferrero-tracking-db pg_dump -U ferrero_user ferrero_tracking | \
gzip > manual/emergency_before_restore_$(date +%Y-%m-%d_%H-%M).sql.gz
# Restore from last known good backup
../database/restore.sh dumps/ferrero_tracking_2025-11-11_02-00.sql.gz
Documentation
For complete backup/restore procedures, see:
- DATABASE_BACKUP_GUIDE.md - Comprehensive guide
- database/backup.sh - Backup script source
- database/restore.sh - Restore script source
- database/check_backups.sh - Health check script
Support
- Backup logs:
logs/backup.log - Restore logs:
logs/restore.log - Health check logs:
logs/backup_check.log - Email notifications: Sent to
ERROR_EMAILon failures
Last Updated: November 11, 2025 Backup System Version: 1.0 Database: ferrero_tracking (PostgreSQL 15)