- Add wrike-monitor-lgl.service for systemd deployment - Add INSTALLATION_LGL.md with complete setup instructions - Include service management commands - Document troubleshooting steps - Add duplicate detection monitoring guide |
||
|---|---|---|
| .gitignore | ||
| config_lgl_team.py | ||
| discover_board_info.py | ||
| INSTALLATION.md | ||
| INSTALLATION_LGL.md | ||
| QUICKSTART.md | ||
| README.md | ||
| requirements.txt | ||
| test_duplicate_detection.py | ||
| wrike-monitor-lgl.service | ||
| wrike-monitor.service | ||
| wrike_monitor.py | ||
| wrike_monitor_lgl.py | ||
Wrike Import Tools
Python tools for automatically importing project structures and deliverable tasks into Wrike from JSON files.
Tools Overview
1. wrike_monitor.py - Real-time Monitor Service (Staging)
A continuous monitoring service that watches a folder 24/7, processes files as they arrive, and sends daily email reports. Designed to run as a systemd service on the Staging board.
2. wrike_monitor_lgl.py - Real-time Monitor Service (LGL Team - Production)
Production version configured for the LGL Team board. Features recursive duplicate detection across the entire Business Areas folder and stores OMG# as HTML links matching the existing board format.
3. discover_board_info.py - Board Discovery Tool
Automated tool to discover Space IDs, custom field IDs, and item types for any Wrike board. Generates configuration snippets for easy setup.
Common Features
Both tools create the following Wrike structure:
- Folders (product categories like "Dry Specialty", "Air", "PDC")
- Projects (campaigns with timelines and metadata)
- Deliverables (created as projects with custom fields and dates)
Both intelligently handle duplicates by checking OMG numbers before creating new items.
Known Limitations
⚠️ Custom Item Type Display: Due to Wrike API v4 limitations, deliverables are created with the correct structure and fields but may display as "Project" type in the UI instead of "Deliverable" type. The customItemTypeId parameter is set in the API, but Wrike UI doesn't always recognize it. If needed, you can manually change the item type in the Wrike UI after creation. All functionality (fields, dates, duplicate detection) works correctly regardless of display type.
Features Comparison
wrike_import.py (Batch Script)
- ✅ Simple command-line usage
- ✅ Process all files and exit
- ✅ Good for one-time imports or manual runs
- ✅ Moves processed files to Processed/
- ✅ Auto-cleanup of old files
wrike_monitor.py (Service)
- ✅ Real-time folder monitoring with watchdog
- ✅ Automatic processing as files arrive
- ✅ Daily email reports at 7PM
- ✅ Periodic scanning for missed files
- ✅ Failed file handling with error logs
- ✅ Auto-cleanup of processed files (24h)
- ✅ Statistics tracking and performance monitoring
- ✅ Systemd service support for production
- ✅ Sequential processing to prevent race conditions
Common Features (Both)
- ✅ Automatic folder creation from BusinessArea hierarchy
- ✅ Project management with start/end dates and campaign codes
- ✅ Deliverable creation as projects with comprehensive custom fields
- ✅ OMG# field with plain job number for duplicate detection
- ✅ OMG URL field with clickable link to OMG system
- ✅ Duplicate prevention via OMG number checking
- ✅ Smart caching to minimize API calls
- ✅ Detailed logging with progress tracking
Quick Start
For Batch Processing (wrike_import.py)
# Install dependencies
pip install requests
# Run on a folder
python wrike_import.py /path/to/json/files/
For Real-time Monitoring (wrike_monitor.py)
# Install dependencies
pip install -r requirements.txt
# Configure paths in wrike_monitor.py (lines 41-44)
# Run the monitor
python wrike_monitor.py
See INSTALLATION.md for systemd service setup.
Requirements
Python Dependencies
For wrike_import.py:
pip install requests
For wrike_monitor.py:
pip install requests watchdog schedule
# Or use requirements.txt:
pip install -r requirements.txt
Python Version
- Python 3.6 or higher
Wrike Requirements
- Valid Wrike API token with write permissions
- Access to the target Wrike space
- Custom fields configured in the space (see Configuration section)
Installation
- Clone or download this repository
- Install dependencies:
pip install requests - Configure your Wrike API token in the script (see Configuration)
Configuration
1. API Token
Edit wrike_import.py and update the WRIKE_TOKEN constant:
WRIKE_TOKEN = "your_wrike_api_token_here"
2. Target Space
Update the STAGING_SPACE_ID to point to your target Wrike space:
STAGING_SPACE_ID = "MQAAAABpz7l_" # Your space ID
To find your space ID:
curl -X GET "https://www.wrike.com/api/v4/spaces" \
-H "Authorization: Bearer YOUR_TOKEN"
3. Custom Fields
The script expects the following custom fields to exist in your Wrike space:
| Field Name | Type | ID Variable | Usage |
|---|---|---|---|
| Budget | Currency | budget |
Project budgets |
| Impact | Dropdown | impact |
Priority level |
| Notes | Text | notes |
Additional information |
| RAG | Dropdown | rag |
Status (Red/Amber/Green) |
| Deliverable Category | Dropdown | deliverable_category |
Type of deliverable |
| Actions | Text | actions |
Next steps |
| Shoot date | Date | shoot_date |
Photography date |
| OMG # | Text | omg_number |
Job/Campaign number |
| Box Link | Text | box_link |
Link to assets |
| Owner | Contacts | owner |
Task owner |
To create these fields, use the Wrike API or create them manually in the Wrike interface.
Update custom field IDs in the script after creating them:
CUSTOM_FIELDS = {
"budget": "YOUR_BUDGET_FIELD_ID",
"impact": "YOUR_IMPACT_FIELD_ID",
# ... etc
}
JSON File Format
Required Structure
{
"JobSpecification": {
"ProjectDetails": {
"BusinessArea": "BISSELL > PRODUCT MARKETING > Dry Specialty",
"Title": "PowerClean FurFinder + FurGuard 2025",
"Description": "<p>PowerClean FurFinder + FurGuard 2025-related workflows</p>",
"StartDate": "2025-04-08 13:00:00+00",
"EndDate": "2025-12-31 17:06:00+00"
},
"JobDetails": {
"Number": "5791330",
"Title": "PowerClean Corded Mass User Guide",
"CampaignCode": "1647476",
"JobCategory": "UX User Guide",
"MediaType": "Creative development",
"Type": "Creative development",
"Notes": "User guide for mass market",
"BriefDate": "2025-05-23 16:00:00+00",
"DueDate": "2025-09-17 16:04:00+00",
"BusinessArea": "BISSELL > PRODUCT MARKETING > Dry Specialty"
}
}
}
Field Mapping
Project Level
| JSON Field | Maps To | Notes |
|---|---|---|
ProjectDetails.BusinessArea |
Folder name | Last segment (e.g., "Dry Specialty") |
ProjectDetails.Title |
Project title | |
ProjectDetails.Description |
Project description | HTML tags removed |
ProjectDetails.StartDate |
Project start date | Converted to YYYY-MM-DD |
ProjectDetails.EndDate |
Project end date | Converted to YYYY-MM-DD |
JobDetails.CampaignCode |
Project OMG # | Custom field |
Task/Deliverable Level
| JSON Field | Maps To | Notes |
|---|---|---|
JobDetails.Number |
Task OMG # | Used for duplicate detection |
JobDetails.Title |
Task title | |
JobDetails.Notes |
Task description | |
JobDetails.JobCategory |
Deliverable Category | Custom field |
JobDetails.MediaType |
Deliverable Category | Fallback if JobCategory empty |
JobDetails.Type |
Notes field | Combined with Details |
JobDetails.BriefDate |
Task start date | |
JobDetails.DueDate |
Task due date |
Usage
Basic Usage
Process all JSON files in a directory:
python wrike_import.py /path/to/json/files/
Example
# Process files in the current directory
python wrike_import.py ./jobs/
# Process files in a specific folder
python wrike_import.py ~/Documents/wrike_imports/
Output Example
Found 3 JSON file(s) to process
Target: Wrike Staging Space (MQAAAABpz7l_)
================================================================================
Processing: 5791330.json
================================================================================
1. Processing folder: 'Dry Specialty'
Found existing folder 'Dry Specialty': MQAAAABpz123
2. Processing project: 'PowerClean FurFinder + FurGuard 2025'
Campaign Code: 1647476
Found existing project 'PowerClean FurFinder + FurGuard 2025': MQAAAABpz456
3. Processing deliverable task
Checking for existing task with OMG #: 5791330
Creating new task 'PowerClean Corded Mass User Guide' (Job #5791330)
✓ Created task 'PowerClean Corded Mass User Guide': MAAAAABpz789
✓ Successfully processed 5791330.json
→ Moved to: Processed/5791330.json
================================================================================
Processing: 5791331.json
================================================================================
1. Processing folder: 'Air'
Found existing folder 'Air': MQAAAABpz124
2. Processing project: 'Air Purifier 2025'
Campaign Code: 1647477
Found existing project 'Air Purifier 2025': MQAAAABpz457
3. Processing deliverable task
Checking for existing task with OMG #: 5791331
Found existing task: MAAAAABpz790
⊙ Task 'Air Purifier Hero Image' already exists (Job #5791331) - skipping
⊙ Successfully processed 5791331.json (task already exists)
→ Moved to: Processed/5791331.json
================================================================================
CLEANUP
================================================================================
Deleted old file: 5791329.json
Deleted old file: 5791328.json
Deleted 2 file(s) older than 24 hours from Processed folder
================================================================================
SUMMARY
================================================================================
Total files: 3
Successful: 3
Skipped (already exists): 1
Failed: 0
Moved to Processed: 3
Folders created/found: 2
Projects created/found: 3
How It Works
1. Folder Management
- Extracts the last segment from
BusinessArea(e.g., "BISSELL > PRODUCT MARKETING > Dry Specialty" → "Dry Specialty") - Checks if folder exists in the target space
- Creates folder if not found
- Caches folder IDs for performance
2. Project Management
- Searches for existing project by title in the folder
- Creates new project if not found
- Converts folder to project with start/end dates
- Adds campaign code as OMG # custom field
- Caches project IDs for performance
3. Task Management
- Checks for existing task by OMG # (job number)
- If found: Updates existing task with new data
- If not found: Creates new task
- Populates all custom fields and dates
4. Duplicate Handling
The script prevents duplicates using OMG numbers:
- Projects: Matched by title within folder
- Tasks: Matched by OMG # custom field (job number)
When processing the same JSON file twice:
- 1st run: Creates folder, project, and task
- 2nd run:
- Finds existing folder and project (reuses them)
- Finds existing task with same OMG # → Skips task creation
- Marks file as "skipped" and moves to Processed folder
Important: Tasks are never updated once created. If a task with the same OMG # exists, it's left unchanged.
5. File Management
After successful processing:
- Moves file to
Processed/subfolder (auto-created) - Failed files remain in the source directory for retry
- Cleanup: Deletes files older than 24 hours from Processed folder
Directory structure:
json_files/
├── 5791331.json # Pending
├── 5791332.json # Pending
└── Processed/
├── 5791330.json # Processed today (kept)
└── 5791329.json # Processed >24h ago (deleted)
Troubleshooting
Common Issues
1. Authentication Error
Error making Wrike request: 401 Unauthorized
Solution: Verify your API token is correct and has write permissions.
2. Custom Field Not Found
Error: Custom field ID not found
Solution: Ensure all custom fields are created in Wrike and IDs are updated in the script.
3. No JSON Files Found
No JSON files found in '/path/to/directory'
Solution:
- Verify the directory path is correct
- Ensure files have
.jsonextension - Check file permissions
4. Invalid Space ID
Error: Space 'MQAAAABpz7l_' not found
Solution:
- Get your space ID from the API:
GET /spaces - Update
STAGING_SPACE_IDin the script
Debug Mode
To see detailed API responses, modify the make_wrike_request function to print full responses:
def make_wrike_request(method, endpoint, data=None):
# ... existing code ...
print(f"Response: {response.json()}") # Add this line
return response.json()
API Rate Limits
Wrike API has rate limits:
- 100 requests per minute per token
- 1000 requests per hour per token
The script is optimized with caching to minimize API calls, but for very large batches (100+ files), you may need to:
- Process in smaller batches
- Add delays between requests
Data Mapping Reference
Date Format Conversion
Input: "2025-05-23 16:00:00+00"
Output: "2025-05-23"
HTML Cleaning
Input: "<p>PowerClean FurFinder + FurGuard 2025-related workflows</p>"
Output: "PowerClean FurFinder + FurGuard 2025-related workflows"
Business Area Parsing
Input: "BISSELL > PRODUCT MARKETING > Dry Specialty"
Output: "Dry Specialty" (folder name)
Advanced Usage
Custom Field Mapping
To add more custom fields, update the script:
- Add field ID to
CUSTOM_FIELDSdictionary - Update
create_or_update_deliverable_task()function:
# Add your custom field
custom_fields.append({
"id": CUSTOM_FIELDS["your_field_name"],
"value": job_details.get("YourJsonField", "")
})
Filter by Business Area
To process only specific business areas, add a filter in process_json_file():
folder_name = parse_business_area(business_area)
# Only process specific folders
if folder_name not in ["Dry Specialty", "Air"]:
print(f" Skipping folder '{folder_name}'")
return False
Batch Processing Script
Create a wrapper script for automated processing:
#!/bin/bash
# process_all.sh
IMPORT_DIR="/path/to/json/files"
# Run import (automatically moves to Processed/ and cleans up old files)
python wrike_import.py "$IMPORT_DIR"
# Optional: Send notification
if [ $? -eq 0 ]; then
echo "Wrike import completed successfully"
fi
Scheduled Processing with Cron
Set up automatic processing every hour:
# Edit crontab
crontab -e
# Add this line to run every hour
0 * * * * /usr/bin/python3 /path/to/wrike_import.py /path/to/json/files >> /var/log/wrike_import.log 2>&1
Change Cleanup Retention Period
To change the 24-hour retention to a different period, modify the cleanup call in the script:
# Delete files older than 48 hours instead
deleted_count = cleanup_old_files(json_dir, hours=48)
# Delete files older than 7 days
deleted_count = cleanup_old_files(json_dir, hours=168)
Security Notes
⚠️ Important Security Considerations:
- Never commit API tokens to version control
- Store tokens in environment variables:
import os WRIKE_TOKEN = os.environ.get('WRIKE_API_TOKEN') - Use a
.envfile for local development (add to.gitignore) - Rotate tokens regularly
- Use read-only tokens for testing
Support
Getting Help
- Check the Wrike API Documentation
- Review error messages in script output
- Enable debug mode for detailed logging
- Check Wrike API status at status.wrike.com
Reporting Issues
When reporting issues, include:
- Script version
- Python version (
python --version) - Full error message
- Sample JSON structure (redacted)
- Wrike space configuration
License
This script is provided as-is for internal use.
Which Tool Should I Use?
Use wrike_import.py if you want to:
- Import a batch of JSON files once
- Run manually when needed
- Test imports locally
- Simple command-line operation
Use wrike_monitor.py if you want to:
- Continuous 24/7 monitoring
- Automatic processing as files arrive
- Daily email reports
- Production deployment as a service
- Detailed statistics and monitoring
LGL Team Production Version (wrike_monitor_lgl.py)
Key Differences from Staging Version
OMG# Format:
- LGL Team stores OMG# as HTML links instead of plain text
- Format:
<a href="https://bissell.omg.oliver.solutions/projects/{id}">{number}</a> - Script automatically generates and parses HTML format
Duplicate Detection:
- Recursive search across entire "Business Areas" folder
- Uses
descendants=trueAPI parameter for efficient searching - Searches ALL subfolders at any depth
- If OMG# exists anywhere in Business Areas → Skip (no duplicate)
- Other folders (Templates, Production Tracker, etc.) are ignored
Configuration:
- API Token: Updated for production access
- Space: "LGL Team" (MQAAAABoHcTY)
- All 11 custom fields mapped to LGL Team IDs
- Custom item types not required (set to None)
Running the LGL Team Version:
# Install dependencies
pip install -r requirements.txt
# Configure paths in wrike_monitor_lgl.py (lines 41-44)
# Run the monitor
python wrike_monitor_lgl.py
Discovery Tool Usage
To discover configuration for any board:
# Run discovery tool
python discover_board_info.py
# Edit TARGET_SPACE in the script to discover different boards
# Generates config_{space_name}.py with all IDs
Changelog
Version 3.0 (December 2025) - LGL Team Migration
- NEW: Added wrike_monitor_lgl.py for LGL Team (production) board
- NEW: HTML link format for OMG# field (matching existing entries)
- NEW: Recursive duplicate detection across entire Business Areas folder
- NEW: discover_board_info.py - automated board configuration discovery
- NEW: HTML extraction and comparison for accurate duplicate detection
- NEW: Global OMG# uniqueness within Business Areas folder
- IMPROVED: Duplicate detection now searches recursively, not just parent project
- IMPROVED: OMG# format matches existing board entries (clickable links)
- FIXED: Handles OMG# stored as HTML
<a>tags - ADDED: config_lgl_team.py - LGL Team configuration reference
- ADDED: test_duplicate_detection.py - testing tool
Version 2.0
- NEW: Added wrike_monitor.py - real-time monitoring service
- NEW: Folder watching with watchdog
- NEW: Daily email reports at 7PM
- NEW: Failed file handling with error logs
- NEW: Periodic scanning for missed files
- NEW: Statistics tracking and performance monitoring
- NEW: systemd service support
- FIXED: Sequential processing to prevent race conditions
- FIXED: Proper parent/child folder matching to avoid duplicates
- IMPROVED: Logging for skipped deliverables
Version 1.2
- Added automatic file movement to Processed subfolder
- Implemented auto-cleanup of files older than 24 hours
- Improved file management and organization
- Enhanced output with file movement tracking
Version 1.1
- Added duplicate detection via OMG numbers
- Separated project and task OMG numbers
- Improved error handling
- Added caching for performance
- Enhanced logging output
Version 1.0
- Initial release
- Basic folder/project/task creation
- Custom field mapping
Last Updated: October 2025 Author: Dave Porter Repository: https://bitbucket.org/zlalani/bissell-wrike-python