obsidian/01 Projects/DevOps_Click_UP_sync/DevOps ClickUp Sync.md

name	client	status	tech	local_path	deploy	url	tags	created	server	port	db
DevOps ↔ ClickUp Sync	Oliver Internal	active	Python FastAPI SQLAlchemy SQLite Docker Pydantic HTTPX	/Users/ai_leed/Documents/Projects/Oliver/DevOps_Click_UP_sync	docker compose up -d --build	http://localhost:8080	oliver devops ado clickup sync webhook	2026-04-14	local	8080	SQLite

Overview

DevOps_Click_UP_sync is a FastAPI webhook service that bi-directionally synchronises Azure DevOps work items with ClickUp tasks in real time. It creates, updates, and mirrors comments and attachments across both platforms, preventing echo-event loops through deduplication logic. The service exposes a simple web dashboard for managing project mappings and viewing sync history, and is deployable as a Docker container with SQLite persistence.

Tech Stack

• Frontend: Vanilla JavaScript + HTML + Tailwind CSS (dashboard at frontend/index.html)
• Backend: Python 3.x + FastAPI + Uvicorn + Pydantic
• Database: SQLite with SQLAlchemy ORM (async via aiosqlite)
• Infrastructure: Docker Compose (single service, volume-mounted data directory)
• AI/ML: N/A
• Key libraries: httpx (async HTTP client for ADO/ClickUp REST APIs), markdownify (HTML↔Markdown conversion), python-dotenv (env config), aiofiles (async file I/O)

Architecture

The service is a synchronisation engine that listens for webhooks from both Azure DevOps and ClickUp, then mirrors changes bidirectionally:

1. Webhook Handlers (src/api/webhooks.py):

   • POST /webhooks/ado — receives ADO work item events
   • POST /webhooks/clickup — receives ClickUp task events
   • HMAC validation via WEBHOOK_SECRET prevents spoofing

2. Sync Engine (src/sync/engine.py):

   • Core logic that determines what to create, update, or skip
   • Calls Field Mapper to translate between ADO and ClickUp schemas
   • Invokes REST clients to push changes back to origin system

3. Field Mapper (src/sync/mapper.py):

   • Transforms ADO work item fields ↔ ClickUp task fields
   • Handles title, description, state, assignee, priority, custom fields
   • Converts HTML ↔ Markdown for comment bodies

4. Echo-Event Deduplication (src/sync/dedup.py):

   • Maintains a dedup table: tracks (entity_id, direction, timestamp)
   • Incoming webhook is skipped if the same entity was synced from the opposite direction within a TTL window (prevents A→B→A loops)

5. REST Clients:

   • src/clients/ado.py — Azure DevOps REST API (work items, comments, attachments)
   • src/clients/clickup.py — ClickUp API (tasks, comments, attachments)

6. Database Models (src/database.py):

   • SyncMap — maps ADO work item ID ↔ ClickUp task ID
   • ProjectMap — maps ADO project ↔ ClickUp space/folder
   • CommentMap — maps ADO comment ↔ ClickUp comment for updates
   • DedupLog — echo-event prevention records

7. Dashboard APIs (src/api/dashboard.py):

   • GET /api/dashboard/stats — sync counts and health
   • GET /api/dashboard/project-maps — view configured mappings
   • GET /api/dashboard/sync-log — paginated sync history

8. Setup API (src/api/setup.py):

   • POST /api/setup/register-webhooks — auto-registers webhook subscriptions with ADO/ClickUp

┌─────────────┐                          ┌──────────────┐
│   Azure     │                          │   ClickUp    │
│   DevOps    │                          │              │
└──────┬──────┘                          └──────┬───────┘
       │                                        │
       │ POST /webhooks/ado                    │ POST /webhooks/clickup
       │ (work item created/updated)          │ (task created/updated)
       │                                        │
       └────────────┬─────────────────────────┘
                    │
            ┌───────▼────────┐
            │   FastAPI      │
            │   Service      │
            │  (port 8080)   │
            └───────┬────────┘
                    │
       ┌────────────┼────────────┐
       │            │            │
       │   Sync     │  Dedup     │
       │  Engine    │  Check     │
       │            │            │
       └──┬─────────┴────────┬───┘
          │                  │
    ┌─────▼──────┐    ┌──────▼────┐
    │   Field    │    │ Database  │
    │  Mapper    │    │ (SQLite)  │
    │            │    │           │
    └─────┬──────┘    └──────┬────┘
          │                  │
    ┌─────▼──────────────────▼────┐
    │   REST Clients               │
    │  (ADO + ClickUp APIs)        │
    └──────────────────────────────┘

Dev Commands

# Copy and configure environment variables
cp .env.example .env
# → Edit .env with ADO_PAT, CLICKUP_API_TOKEN, WEBHOOK_SECRET, PUBLIC_URL, etc.

# Start service in Docker (builds image, mounts ./data volume, runs on port 8080)
docker compose up -d

# Rebuild after code changes
docker compose up -d --build

# View live logs
docker compose logs -f sync-service

# Health check
curl http://localhost:8080/api/health

# Run locally without Docker (requires venv + pip install -r requirements.txt)
pip install -r requirements.txt
uvicorn src.main:app --host 0.0.0.0 --port 8080 --reload

# Access dashboard in browser
http://localhost:8080/

Deployment

• Server: Unknown (no server hostname provided in config)
• Deploy: docker compose up -d --build (from project directory)
• URL: Controlled by PUBLIC_URL environment variable (must be HTTPS and reachable by ADO and ClickUp for webhook callbacks)
• Port: 8080
• Service: Docker Compose service named sync-service (not a systemd service)
• Local path: /Users/ai_leed/Documents/Projects/Oliver/DevOps_Click_UP_sync
• Data persistence: SQLite database stored in ./data/sync.db (Docker volume ./data:/app/data)
• Health check: Built-in Docker healthcheck (GET /api/health every 30s)

Environment Variables

• ADO_ORGANIZATION — Azure DevOps organization name (required)
• ADO_PROJECT — Azure DevOps project name (required)
• ADO_PAT — Azure DevOps Personal Access Token with work item read/write scopes (required)
• ADO_ASSIGNED_TO — WIQL filter for assigned-to field; leave empty to sync all items, use @Me to sync all (optional)
• CLICKUP_API_TOKEN — ClickUp API token (format: pk_...); required
• CLICKUP_WORKSPACE_ID — ClickUp workspace ID (required)
• WEBHOOK_SECRET — Random secret for HMAC signature validation of incoming webhooks (required; must not be changeme or echo-event dedup will not activate)
• PUBLIC_URL — Public HTTPS URL where this service is reachable (required for webhook registration; e.g., https://your-domain.com)
• LOG_LEVEL — Logging level for uvicorn/app (DEBUG, INFO, WARNING, ERROR; default: INFO)
• DATABASE_URL — SQLAlchemy database URL (default: sqlite+aiosqlite:///./data/sync.db); supports PostgreSQL or MySQL if changed

API / Endpoints

Webhooks (External)

• POST /webhooks/ado — Azure DevOps webhook receiver (validates HMAC with WEBHOOK_SECRET)
• POST /webhooks/clickup — ClickUp webhook receiver (validates HMAC with WEBHOOK_SECRET)

Setup

• POST /api/setup/register-webhooks — Automatically registers webhook subscriptions with ADO and ClickUp (idempotent)

Dashboard

• GET / — Serve frontend/index.html (vanilla JS dashboard)
• GET /api/health — Health check endpoint (used by Docker healthcheck)
• GET /api/dashboard/stats — Sync statistics (total synced, last sync time, error count)
• GET /api/dashboard/project-maps — List all configured ADO↔ClickUp project mappings
• GET /api/dashboard/sync-log?page=1&limit=20 — Paginated sync history log

Config

• Routes defined in src/main.py (routers from webhooks.py, setup.py, dashboard.py)

Known Issues

• No test suite — no unit or integration tests configured; no linting setup
• Production hardening needed — before deploying to production:
  • Add pytest + test coverage
  • Add black/flake8 linting
  • Validate all exception handling (especially in sync engine)
  • Add request rate-limiting
  • Implement robust logging/monitoring
• SSH access constraint — no SSH to server without explicit user instruction
• Field mapping gaps — some custom fields may not be handled; ADO→ClickUp HTML→Markdown conversion may lose formatting in edge cases
• **No automatic webhook re-registration

Sessions

2026-04-14 – Project catalogued

Done: Added to Obsidian second brain.

---

Change Log

Date	Requested	Changed	Files
2026-04-14	Initial setup	Note created	—