255c03b65d
Major additions:
- wrike_monitor.py: Real-time folder monitoring with watchdog
- Daily email reports at 7PM with comprehensive statistics
- Failed file handling with error logs
- Periodic scanning for missed files
- systemd service support for production deployment
- Sequential processing to prevent race conditions
- Proper parent/child folder matching using childIds
Configuration:
- Easy path configuration for local/production
- Configurable Wrike space ID
- Email settings for daily reports
- Auto-cleanup of processed files (24h retention)
Documentation:
- INSTALLATION.md: Complete systemd service setup guide
- QUICKSTART.md: Quick reference for both tools
- Updated README.md with tool comparison
- requirements.txt with clear dependencies
Bug fixes:
- Fixed duplicate folder/project creation via childIds matching
- Added logging for skipped deliverables
- Improved cache management
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
359 lines
7.2 KiB
Markdown
359 lines
7.2 KiB
Markdown
# Wrike Monitor Installation Guide
|
|
|
|
## Overview
|
|
|
|
This guide covers installing `wrike_monitor.py` as a systemd service that runs automatically on server boot, monitors a folder 24/7, and sends daily email reports at 7PM.
|
|
|
|
**For simple batch processing**, use `wrike_import.py` instead (no installation needed, just run it).
|
|
|
|
## Prerequisites
|
|
|
|
- Python 3.6+
|
|
- Root/sudo access
|
|
- Wrike API token with write permissions
|
|
|
|
## Installation Steps
|
|
|
|
### 1. Install Python Dependencies
|
|
|
|
```bash
|
|
pip3 install -r requirements.txt
|
|
```
|
|
|
|
Or manually:
|
|
```bash
|
|
pip3 install requests watchdog schedule
|
|
```
|
|
|
|
### 2. Configure the Script
|
|
|
|
Edit `wrike_monitor.py` and update the `Config` class (starting at line 36):
|
|
|
|
#### A. Set Paths (Lines 41-44)
|
|
|
|
**For Local Testing:**
|
|
```python
|
|
HOT_FOLDER = Path("/Users/daveporter/Desktop/CODING-2024/Wrike/json_files")
|
|
PROCESSED_FOLDER = Path("/Users/daveporter/Desktop/CODING-2024/Wrike/Processed")
|
|
FAILED_FOLDER = Path("/Users/daveporter/Desktop/CODING-2024/Wrike/Processed/Failed")
|
|
REPORTS_DIR = Path("/Users/daveporter/Desktop/CODING-2024/Wrike/Processed/reports")
|
|
```
|
|
|
|
**For Production Server:**
|
|
```python
|
|
HOT_FOLDER = Path("/data/PRODUCTION/WRIKE_JSON")
|
|
PROCESSED_FOLDER = Path("/data/PRODUCTION/WRIKE_JSON/Processed")
|
|
FAILED_FOLDER = Path("/data/PRODUCTION/WRIKE_JSON/Failed")
|
|
REPORTS_DIR = Path("/PRODUCTION/WRIKE_LOGS")
|
|
```
|
|
|
|
#### B. Set Wrike Space (Lines 52-53)
|
|
|
|
**For Staging:**
|
|
```python
|
|
WRIKE_SPACE_ID = "MQAAAABpz7l_"
|
|
WRIKE_SPACE_NAME = "Staging"
|
|
```
|
|
|
|
**For Production:**
|
|
```python
|
|
WRIKE_SPACE_ID = "MQAAAABoHcTY"
|
|
WRIKE_SPACE_NAME = "LGL Team"
|
|
```
|
|
|
|
#### C. Update Email Settings (Lines 79, 84)
|
|
|
|
```python
|
|
REPORT_EMAILS = ["your-email@oliver.agency", "another@oliver.agency"]
|
|
DAILY_REPORT_TIME = "19:00" # 7PM daily report
|
|
```
|
|
|
|
#### D. Update API Token (Line 49)
|
|
|
|
```python
|
|
WRIKE_TOKEN = "your_production_api_token_here"
|
|
```
|
|
|
|
### 3. Copy Files to Server
|
|
|
|
```bash
|
|
# Create directory
|
|
sudo mkdir -p /root/wrike-import
|
|
sudo chmod 755 /root/wrike-import
|
|
|
|
# Copy script
|
|
sudo cp wrike_monitor.py /root/wrike-import/
|
|
sudo chmod +x /root/wrike-import/wrike_monitor.py
|
|
|
|
# Create required directories
|
|
sudo mkdir -p /data/PRODUCTION/WRIKE_JSON
|
|
sudo mkdir -p /data/PRODUCTION/WRIKE_JSON/Processed
|
|
sudo mkdir -p /data/PRODUCTION/WRIKE_JSON/Failed
|
|
sudo mkdir -p /PRODUCTION/WRIKE_LOGS
|
|
sudo chmod -R 777 /data/PRODUCTION/WRIKE_JSON
|
|
sudo chmod -R 777 /PRODUCTION/WRIKE_LOGS
|
|
```
|
|
|
|
### 4. Install systemd Service
|
|
|
|
```bash
|
|
# Copy service file
|
|
sudo cp wrike-monitor.service /etc/systemd/system/
|
|
|
|
# Reload systemd
|
|
sudo systemctl daemon-reload
|
|
|
|
# Enable service (start on boot)
|
|
sudo systemctl enable wrike-monitor
|
|
|
|
# Start service
|
|
sudo systemctl start wrike-monitor
|
|
```
|
|
|
|
### 5. Verify Installation
|
|
|
|
```bash
|
|
# Check service status
|
|
sudo systemctl status wrike-monitor
|
|
|
|
# View real-time logs
|
|
sudo journalctl -u wrike-monitor -f
|
|
|
|
# Check if monitoring is working
|
|
echo '{"test": "data"}' > /data/PRODUCTION/WRIKE_JSON/test.json
|
|
sudo journalctl -u wrike-monitor -f
|
|
```
|
|
|
|
## Service Management Commands
|
|
|
|
### Check Status
|
|
```bash
|
|
sudo systemctl status wrike-monitor
|
|
```
|
|
|
|
### Start Service
|
|
```bash
|
|
sudo systemctl start wrike-monitor
|
|
```
|
|
|
|
### Stop Service
|
|
```bash
|
|
sudo systemctl stop wrike-monitor
|
|
```
|
|
|
|
### Restart Service
|
|
```bash
|
|
sudo systemctl restart wrike-monitor
|
|
```
|
|
|
|
### View Logs
|
|
```bash
|
|
# Live tail
|
|
sudo journalctl -u wrike-monitor -f
|
|
|
|
# Last 100 lines
|
|
sudo journalctl -u wrike-monitor -n 100
|
|
|
|
# Since today
|
|
sudo journalctl -u wrike-monitor --since today
|
|
|
|
# Specific time range
|
|
sudo journalctl -u wrike-monitor --since "2025-10-09 09:00" --until "2025-10-09 17:00"
|
|
```
|
|
|
|
### Disable Service
|
|
```bash
|
|
sudo systemctl disable wrike-monitor
|
|
sudo systemctl stop wrike-monitor
|
|
```
|
|
|
|
## Configuration Changes
|
|
|
|
After modifying `wrike_monitor.py`:
|
|
|
|
```bash
|
|
# Restart service to apply changes
|
|
sudo systemctl restart wrike-monitor
|
|
|
|
# Verify it started correctly
|
|
sudo systemctl status wrike-monitor
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Service Won't Start
|
|
|
|
1. Check Python path:
|
|
```bash
|
|
which python3
|
|
# Update ExecStart in wrike-monitor.service if different
|
|
```
|
|
|
|
2. Check file permissions:
|
|
```bash
|
|
ls -la /root/wrike-import/wrike_monitor.py
|
|
sudo chmod +x /root/wrike-import/wrike_monitor.py
|
|
```
|
|
|
|
3. Check logs:
|
|
```bash
|
|
sudo journalctl -u wrike-monitor -n 50
|
|
```
|
|
|
|
### No Files Being Processed
|
|
|
|
1. Verify folder exists and has correct permissions:
|
|
```bash
|
|
ls -la /data/PRODUCTION/WRIKE_JSON
|
|
sudo chmod 777 /data/PRODUCTION/WRIKE_JSON
|
|
```
|
|
|
|
2. Test manually:
|
|
```bash
|
|
sudo systemctl stop wrike-monitor
|
|
cd /root/wrike-import
|
|
python3 wrike_monitor.py
|
|
# Watch for errors, then Ctrl+C
|
|
sudo systemctl start wrike-monitor
|
|
```
|
|
|
|
### Email Reports Not Sending
|
|
|
|
1. Check SMTP credentials in Config class
|
|
2. Verify email addresses in `REPORT_EMAILS`
|
|
3. Check logs for email errors:
|
|
```bash
|
|
sudo journalctl -u wrike-monitor | grep -i email
|
|
```
|
|
|
|
### High CPU/Memory Usage
|
|
|
|
1. Reduce batch size and workers in Config:
|
|
```python
|
|
BATCH_SIZE = 3
|
|
MAX_WORKERS = 3
|
|
```
|
|
|
|
2. Increase scan interval:
|
|
```python
|
|
PERIODIC_SCAN_INTERVAL = 120 # Scan every 2 minutes instead of 1
|
|
```
|
|
|
|
## Testing Before Production
|
|
|
|
### Local Testing
|
|
|
|
1. Keep local paths in Config
|
|
2. Run manually:
|
|
```bash
|
|
python3 wrike_monitor.py
|
|
```
|
|
|
|
3. Drop test JSON files into the hot folder
|
|
4. Verify they're processed and moved to Processed/
|
|
|
|
### Server Testing (without service)
|
|
|
|
```bash
|
|
# Run in foreground
|
|
sudo python3 /root/wrike-import/wrike_monitor.py
|
|
|
|
# Test file processing
|
|
sudo cp test.json /data/PRODUCTION/WRIKE_JSON/
|
|
|
|
# Watch logs
|
|
# Ctrl+C to stop when satisfied
|
|
```
|
|
|
|
## Monitoring Production
|
|
|
|
### Check Daily Reports
|
|
|
|
```bash
|
|
# View today's report
|
|
cat /PRODUCTION/WRIKE_LOGS/daily_report_$(date +%Y-%m-%d).txt
|
|
|
|
# List all reports
|
|
ls -lh /PRODUCTION/WRIKE_LOGS/
|
|
```
|
|
|
|
### Monitor Performance
|
|
|
|
```bash
|
|
# View stats every minute (in logs)
|
|
sudo journalctl -u wrike-monitor -f | grep "📊"
|
|
|
|
# Check for errors
|
|
sudo journalctl -u wrike-monitor | grep -i error
|
|
|
|
# Check for slow scans
|
|
sudo journalctl -u wrike-monitor | grep -i "slow"
|
|
```
|
|
|
|
### Check Processed/Failed Files
|
|
|
|
```bash
|
|
# Processed files (will auto-delete after 24h)
|
|
ls -lh /data/PRODUCTION/WRIKE_JSON/Processed/
|
|
|
|
# Failed files (need manual review)
|
|
ls -lh /data/PRODUCTION/WRIKE_JSON/Failed/
|
|
|
|
# View error logs
|
|
cat /data/PRODUCTION/WRIKE_JSON/Failed/*_ERROR.txt
|
|
```
|
|
|
|
## Maintenance
|
|
|
|
### Manually Trigger Daily Report
|
|
|
|
```bash
|
|
# The report runs automatically at 7PM, but to test:
|
|
sudo systemctl restart wrike-monitor
|
|
# Then wait or modify DAILY_REPORT_TIME in config
|
|
```
|
|
|
|
### Clear Failed Files After Review
|
|
|
|
```bash
|
|
# After reviewing and fixing failed files
|
|
sudo rm -rf /data/PRODUCTION/WRIKE_JSON/Failed/*
|
|
```
|
|
|
|
### Backup Configuration
|
|
|
|
```bash
|
|
# Backup before changes
|
|
sudo cp /root/wrike-import/wrike_monitor.py /root/wrike-import/wrike_monitor.py.backup
|
|
```
|
|
|
|
## Uninstallation
|
|
|
|
```bash
|
|
# Stop and disable service
|
|
sudo systemctl stop wrike-monitor
|
|
sudo systemctl disable wrike-monitor
|
|
|
|
# Remove service file
|
|
sudo rm /etc/systemd/system/wrike-monitor.service
|
|
sudo systemctl daemon-reload
|
|
|
|
# Remove script (optional)
|
|
sudo rm -rf /root/wrike-import
|
|
|
|
# Remove data directories (optional - be careful!)
|
|
# sudo rm -rf /data/PRODUCTION/WRIKE_JSON
|
|
# sudo rm -rf /PRODUCTION/WRIKE_LOGS
|
|
```
|
|
|
|
## Support
|
|
|
|
For issues:
|
|
1. Check logs: `sudo journalctl -u wrike-monitor -f`
|
|
2. Verify configuration in `wrike_monitor.py`
|
|
3. Test manually before running as service
|
|
4. Review daily reports for patterns
|
|
|
|
---
|
|
|
|
**Last Updated**: October 2025
|