vault backup: 2026-04-29 13:24:32
This commit is contained in:
Vadym Samoilenko
parent
0eff2c4e20
commit
affc6a8353
29 changed files with 3757 additions and 450 deletions
|
|
@ -2,64 +2,158 @@
|
|||
name: "3M OMG Portal"
|
||||
client: 3M
|
||||
status: active
|
||||
server: baic (web-03)
|
||||
tech: [Node.js, Vanilla JS, HTML/CSS, One2Edit API]
|
||||
server: local
|
||||
tech: [Node.js, Vanilla JavaScript, HTML/CSS, One2Edit API, CORS Proxy]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/3m-portal
|
||||
deploy: npm start / node server.js
|
||||
deploy: npm start
|
||||
url: http://localhost:3000
|
||||
tags: [3m, one2edit, translation, proxy, nodejs]
|
||||
created: 2026-04-14
|
||||
last_commit: 2026-03-11
|
||||
commits: 7
|
||||
port: 3000
|
||||
---
|
||||
|
||||
## Overview
|
||||
Lightweight portal wrapping the One2Edit API for 3M translation job management. Node.js acts as a CORS proxy to `https://oliver.one2edit.com/v3/Api.php`.
|
||||
|
||||
**Page flow:**
|
||||
1. `login.html` — Two-step auth: username→userId, then externSessionId
|
||||
2. `dashboard.html` — Fetch jobs (STARTED/RUNNING), render progress, PDF export
|
||||
3. `editor.html` — Embed One2Edit JS SDK with externSessionId
|
||||
|
||||
**Two auth modes:**
|
||||
- Credential-based: portal service account (`portal@oliver.agency`) for job listing
|
||||
- Session-based: `externSessionId` for embedded editor
|
||||
The 3M OMG Portal is a lightweight web interface for managing One2Edit translation jobs with dual authentication (Oliver and 3M login options). It serves as a CORS proxy between the browser and the One2Edit API (`https://oliver.one2edit.com/v3/Api.php`), allowing users to view active jobs, track progress, export PDFs, and open jobs in the embedded One2Edit editor. This is a minimal, vanilla JS project with no build step or transpilation.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** Vanilla JS + HTML/CSS (no transpilation, no build step)
|
||||
- **Backend:** Node.js (`server.js`) — static files + `/api` CORS proxy
|
||||
- **API:** One2Edit v3 (`oliver.one2edit.com/v3/Api.php`)
|
||||
- **Session:** `sessionStorage` (cleared on browser close)
|
||||
- **Infrastructure:** No Docker
|
||||
- **Frontend:** Vanilla JavaScript (ES6), HTML5, CSS3
|
||||
- **Backend:** Node.js with Express (implied), serving static files + CORS proxy
|
||||
- **Database:** N/A (stateless; session data in browser `sessionStorage`)
|
||||
- **Infrastructure:** Local Node.js server, no Docker
|
||||
- **API Integration:** One2Edit API (https://oliver.one2edit.com/v3/Api.php)
|
||||
- **Key libraries:** None (vanilla stack)
|
||||
|
||||
## Architecture
|
||||
|
||||
The project follows a simple three-tier architecture:
|
||||
|
||||
```
|
||||
Browser → localhost:3000 (static files)
|
||||
Browser → localhost:3000/api → oliver.one2edit.com/v3/Api.php (CORS proxy)
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ BROWSER (Client) │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ login.html ─auth.js─┐ │
|
||||
│ ├─> sessionStorage (isAuthenticated, │
|
||||
│ dashboard.html ─────┤ username, userId, externSessionId, │
|
||||
│ ─dashboard.js │ authConfig) │
|
||||
│ │ │
|
||||
│ editor.html ────────┘ │
|
||||
│ (One2Edit SDK) │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
│ │
|
||||
│ HTTP requests │ /api proxy calls
|
||||
▼ ▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ NODE.JS SERVER (server.js) │
|
||||
│ localhost:3000 │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ • Static file serving (HTML, CSS, JS, images) │
|
||||
│ • CORS proxy: rewrites Location headers on redirects │
|
||||
│ • Masks passwords in logs │
|
||||
│ • Injects CORS headers on /api responses │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ HTTPS proxied requests
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ ONE2EDIT API (oliver.one2edit.com/v3/Api.php) │
|
||||
│ • user.info (resolve username → userId) │
|
||||
│ • user.session.extern.add (generate externSessionId) │
|
||||
│ • job.list (fetch STARTED/RUNNING jobs) │
|
||||
│ • job.export (PDF generation) │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
**server.js proxy behavior:**
|
||||
- Strips/rewrites Location headers on 301/302 → returns 401 (prevents auth redirect loops)
|
||||
- Injects CORS headers on all /api responses
|
||||
- Masks passwords in logs
|
||||
|
||||
**Authentication Flow:**
|
||||
1. User enters credentials on `login.html`
|
||||
2. `auth.js` calls `user.info` API (credential-based) to resolve username → `userId`
|
||||
3. `auth.js` calls `user.session.extern.add` to generate `externSessionId`
|
||||
4. Session tokens stored in `sessionStorage`; redirects to `dashboard.html`
|
||||
5. When opening a job editor, `externSessionId` is passed to One2Edit SDK
|
||||
|
||||
**Two auth modes:**
|
||||
- **Credential-based:** Portal service account (`portal@oliver.agency`) used for job listing and PDF export
|
||||
- **Session-based:** `externSessionId` passed to embedded One2Edit SDK for per-user editing
|
||||
|
||||
## Dev Commands
|
||||
```bash
|
||||
npm start # or: node server.js → http://localhost:3000
|
||||
# No build step, no test suite
|
||||
npm start # Start server on http://localhost:3000
|
||||
node server.js # Same as above (entry point defined in package.json)
|
||||
```
|
||||
|
||||
## Key Files
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `server.js` | HTTP server: static + /api proxy |
|
||||
| `auth.js` | Login logic (Oliver + 3M auth buttons) |
|
||||
| `dashboard.js` | Job list, filtering, PDF export |
|
||||
| `editor.html` | One2Edit SDK wrapper |
|
||||
No build step, no test suite. Vanilla JS runs directly in the browser.
|
||||
|
||||
## Deployment
|
||||
- **Run:** `node server.js` → http://localhost:3000
|
||||
- **Server:** local (development only; no production deployment documented)
|
||||
- **Deploy:** `npm start` (starts Node.js server)
|
||||
- **URL:** N/A (development: http://localhost:3000)
|
||||
- **Port:** 3000 (configurable via `.env` `PORT` variable)
|
||||
- **Service:** N/A (not deployed as systemd service in current state)
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/3m-portal`
|
||||
|
||||
**Git remote:** `git@bitbucket.org:zlalani/3m-portal.git`
|
||||
|
||||
## Environment Variables
|
||||
- `SERVICE_USERNAME` — Portal service account username (e.g., `portal@oliver.agency`), used for job listing and PDF export
|
||||
- `SERVICE_PASSWORD` — Portal service account password
|
||||
- `PORT` — Server port (default: 3000)
|
||||
|
||||
Reference: `.env.example`
|
||||
|
||||
## Project Structure
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `server.js` | Node.js HTTP server: static serving + `/api` CORS proxy to One2Edit |
|
||||
| `auth.js` | Login handler: credential validation, user.info lookup, session.extern.add, dual auth buttons |
|
||||
| `login.html` | Login page with Oliver/3M branding, username/password form |
|
||||
| `dashboard.html` | Job list view, filtering, progress indicators, PDF export trigger |
|
||||
| `dashboard.js` | Job fetching, rendering, filtering logic, PDF export handler |
|
||||
| `editor.html` | Minimal wrapper that loads One2Edit SDK from CDN, embeds editor with externSessionId |
|
||||
| `api.js` | Legacy API module (not actively used; kept for reference) |
|
||||
| `login.js` | Legacy login handler (superseded by `auth.js`) |
|
||||
| `.env.example` | Template for environment variables |
|
||||
| `package.json` | Node dependencies and npm scripts |
|
||||
|
||||
## API / Endpoints
|
||||
|
||||
**Proxy endpoint:**
|
||||
- `POST /api` — Proxies requests to `https://oliver.one2edit.com/v3/Api.php`
|
||||
|
||||
**One2Edit API calls made by the portal:**
|
||||
- `user.info` — Resolve username to userId (credential-based)
|
||||
- `user.session.extern.add` — Generate externSessionId for embedded editor
|
||||
- `job.list` — Fetch jobs with status STARTED or RUNNING
|
||||
- `job.export` — Export job as PDF
|
||||
|
||||
## Key Features
|
||||
- **Dual authentication:** Oliver login + 3M SSO stubs (MSAL placeholders added in recent commits)
|
||||
- **Job dashboard:** Filter by status, view progress with visual indicators
|
||||
- **One2Edit integration:** Embed editor with external session tokens
|
||||
- **PDF export:** Download job state as PDF blob
|
||||
- **CORS proxy:** Prevents auth redirect loops by rewriting Location headers
|
||||
- **Password masking:** Server logs do not expose credentials
|
||||
|
||||
## Known Issues
|
||||
- `api.js` and `login.js` are legacy and not actively maintained
|
||||
- SSO/MSAL implementation is stubbed; full 3M integration incomplete
|
||||
- No build step or transpilation means limited modern JS framework support
|
||||
- No test suite; manual testing only
|
||||
- Session data cleared on browser close (no persistence layer)
|
||||
|
||||
## Git History
|
||||
Recent commits focus on:
|
||||
- Asset addition (splash image, login logo)
|
||||
- Deployment script cleanup (SSL config removed, git operations delegated to user)
|
||||
- SSO stubs and password field restoration
|
||||
- Production-readiness fixes (remove hardcoded credentials)
|
||||
- Initial portal source files commit
|
||||
|
||||
---
|
||||
|
||||
**For new developers:** Start by running `npm start`, navigate to `http://localhost:3000`, and review `auth.js` and `dashboard.js` for the core logic. The project is intentionally minimal to keep deployment and maintenance friction low.
|
||||
|
||||
## Timeline / Git History
|
||||
| Date | Change |
|
||||
|------|--------|
|
||||
|
|
@ -76,4 +170,4 @@ npm start # or: node server.js → http://localhost:3000
|
|||
| 2026-03-11 | Missing images | Add 3M_Splash.jpg, login_logo.png | assets/ |
|
||||
|
||||
## Related
|
||||
- [[hm-o2e-tool/HM O2E Tool]] (same One2Edit platform)
|
||||
- [[hm-o2e-tool/HM O2E Tool]] (same One2Edit platform)
|
||||
|
|
@ -1,37 +1,126 @@
|
|||
---
|
||||
name: "DevOps ↔ ClickUp Sync"
|
||||
client: Oliver Internal
|
||||
client: Oliver
|
||||
status: active
|
||||
tech: [Python, FastAPI, SQLAlchemy, SQLite, httpx, Tailwind, Docker]
|
||||
tech: [Python, FastAPI, SQLAlchemy, SQLite, Docker, Azure DevOps API, ClickUp API]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/DevOps_Click_UP_sync
|
||||
deploy: docker compose up --build
|
||||
deploy: docker-compose up
|
||||
url: http://localhost:8080
|
||||
tags: [oliver, devops, ado, clickup, sync, webhook]
|
||||
created: 2026-04-14
|
||||
server: local
|
||||
port: 8080
|
||||
db: SQLite
|
||||
---
|
||||
|
||||
## Overview
|
||||
Full two-way sync between Azure DevOps (ADO) and ClickUp. Webhook-driven, real-time. Tailwind CSS monitoring dashboard.
|
||||
DevOps_Click_UP_sync is a bidirectional synchronization service that bridges Azure DevOps and ClickUp, enabling seamless task/work item management across both platforms. It runs as a containerized FastAPI application that listens for webhooks and maintains data consistency between the two systems. The service is designed for teams using both Azure DevOps and ClickUp who need unified task tracking without manual duplication.
|
||||
|
||||
## Tech Stack
|
||||
- **Backend:** Python + FastAPI + SQLAlchemy async
|
||||
- **Database:** SQLite
|
||||
- **HTTP:** httpx (async)
|
||||
- **Frontend:** Vanilla JS + Tailwind CDN
|
||||
- **Infrastructure:** Docker + docker-compose
|
||||
- **Frontend:** N/A (API-only service)
|
||||
- **Backend:** Python 3.x, FastAPI, Uvicorn ASGI server
|
||||
- **Database:** SQLite with SQLAlchemy ORM and async support (aiosqlite)
|
||||
- **Infrastructure:** Docker, Docker Compose
|
||||
- **AI/ML:** N/A
|
||||
- **Key libraries:** FastAPI, SQLAlchemy, httpx (async HTTP client), Pydantic (validation), markdownify (content conversion)
|
||||
|
||||
## Architecture
|
||||
The service follows a webhook-based event-driven architecture:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ External Systems │
|
||||
│ ┌──────────────────┐ ┌──────────────────┐ │
|
||||
│ │ Azure DevOps │ │ ClickUp │ │
|
||||
│ └────────┬─────────┘ └────────┬─────────┘ │
|
||||
└───────────┼────────────────────────┼───────────────┘
|
||||
│ Webhooks │ Webhooks
|
||||
└────────────┬───────────┘
|
||||
│
|
||||
┌────────────────▼─────────────────┐
|
||||
│ FastAPI Sync Service │
|
||||
│ ┌──────────────────────────┐ │
|
||||
│ │ /api/health │ │
|
||||
│ │ /api/webhook/ado │ │
|
||||
│ │ /api/webhook/clickup │ │
|
||||
│ │ /api/sync │ │
|
||||
│ └──────────────────────────┘ │
|
||||
└────────────────┬──────────────────┘
|
||||
│
|
||||
┌────────────────▼─────────────────┐
|
||||
│ SQLite Database (/app/data) │
|
||||
│ - Sync mappings │
|
||||
│ - Metadata │
|
||||
└──────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Key Design Decisions:**
|
||||
- **Async-first:** All database and HTTP operations use async/await for scalability
|
||||
- **Webhook-driven:** Listens for changes from both platforms rather than polling
|
||||
- **Content conversion:** Uses markdownify to normalize formatting between systems
|
||||
- **Health checks:** Built-in health endpoint for container orchestration
|
||||
- **Persistent storage:** SQLite for tracking sync state and mappings
|
||||
|
||||
## Dev Commands
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Create .env file from template
|
||||
cp .env.example .env
|
||||
# Edit .env with your credentials
|
||||
|
||||
# Run locally (requires Python 3.9+)
|
||||
python -m uvicorn main:app --reload --port 8080
|
||||
|
||||
# Build and run with Docker
|
||||
docker-compose up --build
|
||||
|
||||
# View logs
|
||||
docker-compose logs -f sync-service
|
||||
|
||||
# Stop services
|
||||
docker-compose down
|
||||
|
||||
# Access health check
|
||||
curl http://localhost:8080/api/health
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `docker compose up --build`
|
||||
- **Entry:** `src/main.py` (uvicorn, port 8080)
|
||||
- **Config:** Copy `.env.example` → `.env`
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/DevOps_Click_UP_sync`
|
||||
- **Server:** local (intended for container deployment)
|
||||
- **Deploy:** `docker-compose up -d`
|
||||
- **URL:** Configured via `PUBLIC_URL` env var (e.g., https://your-domain.com)
|
||||
- **Port:** 8080
|
||||
- **Service:** Docker container managed by docker-compose
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/DevOps_Click_UP_sync
|
||||
|
||||
## Key Files
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/sync/engine.py` | Core sync logic |
|
||||
| `src/sync/dedup.py` | Loop prevention |
|
||||
| `src/sync/mapper.py` | ADO ↔ ClickUp field mapping |
|
||||
**Container details:**
|
||||
- Health check runs every 30s (timeout 10s, 3 retries)
|
||||
- Auto-restart policy: unless-stopped
|
||||
- Volumes: ./data mounted to /app/data (persistent database storage)
|
||||
|
||||
## Environment Variables
|
||||
- `ADO_ORGANIZATION` — Azure DevOps organization name
|
||||
- `ADO_PROJECT` — Azure DevOps project name
|
||||
- `ADO_PAT` — Azure DevOps Personal Access Token (authentication)
|
||||
- `CLICKUP_API_TOKEN` — ClickUp API token (format: pk_*)
|
||||
- `CLICKUP_WORKSPACE_ID` — ClickUp workspace identifier
|
||||
- `WEBHOOK_SECRET` — Secret key for validating incoming webhooks
|
||||
- `PUBLIC_URL` — Public domain/URL where this service is accessible (for webhook callbacks)
|
||||
- `LOG_LEVEL` — Logging verbosity (INFO, DEBUG, etc.)
|
||||
- `DATABASE_URL` — SQLite connection string; default: `sqlite+aiosqlite:///./data/sync.db`
|
||||
|
||||
## API / Endpoints
|
||||
- `GET /api/health` — Health check endpoint (used by Docker healthcheck)
|
||||
- `POST /api/webhook/ado` — Receives webhook events from Azure DevOps
|
||||
- `POST /api/webhook/clickup` — Receives webhook events from ClickUp
|
||||
- `POST /api/sync` — Manually trigger synchronization (if implemented)
|
||||
|
||||
## Known Issues
|
||||
None documented. Ensure webhook payloads are validated with `WEBHOOK_SECRET` to prevent unauthorized sync triggers.
|
||||
|
||||
## Git
|
||||
- **Remote:** [Not provided in source files]
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
|
|
@ -41,4 +130,4 @@ Full two-way sync between Azure DevOps (ADO) and ClickUp. Webhook-driven, real-t
|
|||
## Change Log
|
||||
| Date | Requested | Changed | Files |
|
||||
|------|-----------|---------|-------|
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
|
|
@ -1,28 +1,144 @@
|
|||
---
|
||||
name: "Oliver AI Bot 2.0 (Nexus MVP)"
|
||||
client: Oliver Internal
|
||||
client: Enterprise
|
||||
status: active
|
||||
tech: [Next.js, FastAPI, Python, Docker, RAG]
|
||||
tech: [React, TypeScript, FastAPI, PostgreSQL, Redis, Qdrant, Docker]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/Oliver-ai-bot_2.0
|
||||
deploy: docker compose up --build
|
||||
url:
|
||||
deploy: docker-compose up -d
|
||||
url: http://localhost:3000
|
||||
tags: [oliver, ai, rag, nexus, executive-assistant, notebook]
|
||||
created: 2026-04-14
|
||||
server: local
|
||||
port: 3000
|
||||
db: PostgreSQL
|
||||
---
|
||||
|
||||
## Overview
|
||||
Enterprise AI Hub "Nexus" MVP. 3 modes: RAG (knowledge retrieval), Executive Assistant (calendar/email/tasks), Notebook.
|
||||
|
||||
Status: MVP Complete — Frontend 100%, Backend 85%
|
||||
**Nexus** (Oliver-ai-bot_2.0) is an enterprise-grade unified AI platform combining RAG (Retrieval-Augmented Generation), an AI Executive Assistant, and a Notebook mode for document analysis. It provides corporate users with a centralized interface to query knowledge bases with citations, perform AI-assisted productivity tasks (summarization, translation, action item extraction), and manage documents. The platform includes Microsoft Entra ID authentication, role-based access control (Super Admin, Content Manager, User), and an admin dashboard for system configuration—currently at MVP completion status running in Docker.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** Next.js
|
||||
- **Backend:** FastAPI + Python
|
||||
- **Infrastructure:** Docker + docker-compose
|
||||
- **Frontend:** React 18+, TypeScript, Vite, TailwindCSS, Shadcn/ui, React Query
|
||||
- **Backend:** FastAPI (Python), Uvicorn, SQLAlchemy ORM
|
||||
- **Database:** PostgreSQL 16, Redis 7 (caching), Qdrant (vector database for RAG)
|
||||
- **Infrastructure:** Docker, Docker Compose, multi-container orchestration
|
||||
- **AI/ML:** OpenAI (GPT-5.1, embeddings), Google Gemini 2.5 Flash, Anthropic Claude Sonnet 4.5, NotebookLlama integration
|
||||
- **Key libraries:** Pydantic (validation), python-jose (JWT), psycopg2 (PostgreSQL driver), aioredis, httpx
|
||||
|
||||
## Architecture
|
||||
Nexus follows a 3-tier microservices architecture with containerized services:
|
||||
|
||||
- **Frontend** (React/TypeScript): SPA running on port 3000, consumes REST API, handles authentication via OAuth2, implements SSE streaming for real-time responses
|
||||
- **Backend** (FastAPI): REST API on port 8000, handles auth (Microsoft Entra ID + JWT), RAG chat logic, document processing, user/admin endpoints, integrates with external LLM providers
|
||||
- **Data Layer:**
|
||||
- PostgreSQL (port 5432): Stores users, conversations, documents, configuration
|
||||
- Redis (port 6379): Session caching, token blacklisting, rate limiting
|
||||
- Qdrant (port 6333): Vector embeddings for RAG semantic search
|
||||
- **External Integrations:** OpenAI API (embeddings & chat), Google AI (assistant tasks), Anthropic API (writing tasks), Microsoft Entra ID (corporate SSO), NotebookLlama (notebook mode)
|
||||
|
||||
Data flow: Frontend sends queries → FastAPI validates & routes → LLM APIs process → Qdrant retrieves similar docs → Response streams back via SSE → Frontend renders with markdown & citations.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Browser (React SPA) │
|
||||
│ http://localhost:3000 │
|
||||
└────────────────────┬────────────────────────────────────┘
|
||||
│ REST + SSE
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ FastAPI Backend │
|
||||
│ http://localhost:8000 │
|
||||
│ ├─ /auth (OAuth2 Entra ID + JWT) │
|
||||
│ ├─ /api/v1/chat (RAG streaming) │
|
||||
│ ├─ /api/v1/assistant (summarize/translate/extract) │
|
||||
│ ├─ /api/v1/admin (user mgmt, config) │
|
||||
│ └─ /docs (Swagger UI) │
|
||||
└──┬──────────────────┬──────────────────┬────────────────┘
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
PostgreSQL Redis Qdrant
|
||||
(Users, (Sessions, (Vector
|
||||
Conversations) Tokens) Embeddings)
|
||||
```
|
||||
|
||||
## Dev Commands
|
||||
```bash
|
||||
# Clone and navigate
|
||||
git clone git@bitbucket.org:zlalani/enterprise-ai-hub-nexus.git
|
||||
cd Oliver-ai-bot_2.0
|
||||
|
||||
# Start all backend services with Docker Compose
|
||||
docker-compose up -d
|
||||
|
||||
# Verify services are healthy
|
||||
docker ps
|
||||
|
||||
# Start frontend development server
|
||||
cd frontend
|
||||
npm install
|
||||
npm run dev
|
||||
|
||||
# Backend API documentation (once running)
|
||||
# Open: http://localhost:8000/docs
|
||||
|
||||
# Stop all services
|
||||
docker-compose down
|
||||
|
||||
# View logs
|
||||
docker-compose logs -f backend
|
||||
docker-compose logs -f db
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `docker compose up --build`
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/Oliver-ai-bot_2.0`
|
||||
- **Server:** local (Docker containers)
|
||||
- **Deploy:** `docker-compose up -d` (starts all services: PostgreSQL, Redis, Qdrant, FastAPI backend)
|
||||
- **URL:** http://localhost:3000 (frontend), http://localhost:8000 (API)
|
||||
- **Port:** 3000 (frontend), 8000 (backend API), 5432 (PostgreSQL), 6379 (Redis), 6333 (Qdrant)
|
||||
- **Service:** Docker Compose managed (no systemd service)
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/Oliver-ai-bot_2.0
|
||||
|
||||
## Environment Variables
|
||||
Key variables (see `.env.example` for complete list):
|
||||
|
||||
- `POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD` — PostgreSQL credentials and database name
|
||||
- `DATABASE_URL` — Full PostgreSQL connection string
|
||||
- `REDIS_URL` — Redis connection (default: redis://localhost:6379/0)
|
||||
- `QDRANT_URL` — Vector DB endpoint (default: http://localhost:6333)
|
||||
- `ENTRA_CLIENT_ID`, `ENTRA_CLIENT_SECRET`, `ENTRA_TENANT_ID` — Microsoft Azure AD OAuth2 credentials
|
||||
- `ENTRA_REDIRECT_URI` — OAuth callback URL (default: http://localhost:8000/api/v1/auth/callback)
|
||||
- `OPENAI_API_KEY` — OpenAI API key (for GPT-5.1 chat & embeddings)
|
||||
- `GOOGLE_API_KEY` — Google AI Studio key (for Gemini 2.5 Flash assistant)
|
||||
- `ANTHROPIC_API_KEY` — Anthropic API key (for Claude Sonnet 4.5 writing)
|
||||
- `JWT_SECRET` — Signing key for JWT tokens (min 32 chars, generate via `openssl rand -hex 32`)
|
||||
- `JWT_ALGORITHM`, `JWT_EXPIRATION_MINUTES`, `REFRESH_TOKEN_EXPIRATION_DAYS` — Token config
|
||||
- `NOTEBOOKLLAMA_URL` — Internal NotebookLlama service endpoint
|
||||
- `MAX_UPLOAD_SIZE_MB` — File upload limit (default: 100 MB)
|
||||
- `ENVIRONMENT` — development | staging | production
|
||||
- `DEBUG` — Enable debug logging
|
||||
|
||||
## API / Endpoints
|
||||
- `POST /api/v1/auth/login` — Microsoft Entra ID OAuth login redirect
|
||||
- `POST /api/v1/auth/callback` — OAuth callback handler
|
||||
- `POST /api/v1/auth/refresh` — Refresh JWT token
|
||||
- `POST /api/v1/chat` — Stream RAG chat responses (SSE)
|
||||
- `GET /api/v1/chat/history` — Retrieve conversation history
|
||||
- `POST /api/v1/assistant/summarize` — Summarize meeting notes
|
||||
- `POST /api/v1/assistant/translate` — Translate document (8 languages)
|
||||
- `POST /api/v1/assistant/extract` — Extract action items
|
||||
- `GET /api/v1/admin/users` — List users (Super Admin only)
|
||||
- `POST /api/v1/admin/config` — Update LLM provider config (Super Admin)
|
||||
- `GET /api/v1/profile` — Get current user profile
|
||||
- `GET /docs` — Swagger API documentation
|
||||
|
||||
## Known Issues
|
||||
- Admin Dashboard Analytics is a placeholder UI (not yet implemented)
|
||||
- Mobile UI optimized but desktop-primary
|
||||
- NotebookLlama integration URL must be configured in `.env`
|
||||
- Frontend at 100% completion, Backend at 85% completion (likely referring to advanced features/edge cases)
|
||||
|
||||
## Git
|
||||
- **Remote:** git@bitbucket.org:zlalani/enterprise-ai-hub-nexus.git
|
||||
- **Latest commits:** Phase 6 complete (Assistant Mode, Admin Dashboard, final polish); MVP running in Docker
|
||||
- **Branch strategy:** Not specified in provided logs
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
|
|
@ -36,4 +152,4 @@ Status: MVP Complete — Frontend 100%, Backend 85%
|
|||
|
||||
## Related
|
||||
- [[enterprise-ai-hub-nexus/Enterprise AI Hub Nexus]]
|
||||
- [[oliver-ai-assistant/Oliver AI Assistant]]
|
||||
- [[oliver-ai-assistant/Oliver AI Assistant]]
|
||||
|
|
@ -1,28 +1,193 @@
|
|||
---
|
||||
name: "Activation Calendar Helper"
|
||||
client: Oliver Internal
|
||||
client: Oliver
|
||||
status: active
|
||||
server: optical-web-1
|
||||
tech: [Python, Docker, Google Gemini, HTML/JS, Tailwind]
|
||||
server: ai-sandbox.oliver.solutions
|
||||
tech: [PHP, JavaScript, PostgreSQL, Docker, Azure AD, OpenAI, Google Gemini, Anthropic Claude]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/ac-helper
|
||||
deploy: docker compose up --build
|
||||
url:
|
||||
deploy: docker-compose up -d
|
||||
url: https://ai-sandbox.oliver.solutions/ac-helper/
|
||||
tags: [oliver, activation-calendar, ai, gemini, deliverables]
|
||||
created: 2026-04-14
|
||||
port: 8100
|
||||
db: PostgreSQL
|
||||
---
|
||||
|
||||
## Overview
|
||||
Marketing deliverables management with AI automation. Natural language commands (Gemini), interactive spreadsheet grid, multi-sheet support, strict 3-level hierarchy (Category → Media → Sub-media), CSV export.
|
||||
|
||||
AC Helper is a web-based marketing deliverables management tool for the Oliver agency that combines a smart spreadsheet interface with AI-powered automation. Users manage activation campaigns across a 3-level hierarchy (Category → Media → Sub-media) with AI commands powered by Gemini, OpenAI, and Claude. The platform supports multi-sheet workspaces, real-time exports, role-based access control via Azure AD SSO, and emergency token-based login for failover access.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** HTML/JS + Tailwind CDN
|
||||
- **Backend:** Python
|
||||
- **AI:** Google Gemini
|
||||
- **Infrastructure:** Docker + docker-compose
|
||||
|
||||
- **Frontend:** JavaScript (Vanilla), Handsontable (spreadsheet grid), MSAL (Azure AD SPA PKCE auth)
|
||||
- **Backend:** PHP (Legacy), Python/FastAPI (New containerized version), asyncpg for async PostgreSQL
|
||||
- **Database:** PostgreSQL 16 Alpine (sheets, metadata, audit logs)
|
||||
- **Infrastructure:** Docker Compose (app + postgres), Apache reverse proxy, environment-based configuration
|
||||
- **AI/ML:** Google Gemini 2.0 Flash, OpenAI GPT-4.1, Anthropic Claude Opus/Sonnet, LlamaCloud (PDF parsing)
|
||||
- **Key Libraries:** Jspreadsheet (older grid), Handsontable (newer grid), axios, dotenv
|
||||
|
||||
## Architecture
|
||||
|
||||
The project is undergoing migration from a legacy PHP file-based system to a modern containerized Python/PostgreSQL stack:
|
||||
|
||||
**Current Production (Hybrid):**
|
||||
- Apache serves static frontend files (`index.html`, `script.js`, `style.css`)
|
||||
- Apache reverse-proxies API requests (`/api/`, `/ws/`) to Docker container on localhost:8100
|
||||
- PostgreSQL runs in a separate Docker service (`ac-tool-db`)
|
||||
- Session management via `SESSION_SECRET` environment variable
|
||||
- Authentication: Azure AD MSAL (primary) + Emergency Token bypass
|
||||
|
||||
**Data Flow:**
|
||||
1. User logs in via Azure AD PKCE flow or emergency token
|
||||
2. Frontend makes API calls to `/api/` endpoints
|
||||
3. Backend queries PostgreSQL for sheets, metadata, and user roles
|
||||
4. AI commands sent to Gemini/OpenAI/Claude for natural language processing
|
||||
5. Results written back to database and real-time updates via websockets
|
||||
|
||||
**Key Design Decisions:**
|
||||
- Strict 3-level hierarchy enforced at DB level (prevents invalid combinations)
|
||||
- Per-client category hierarchies support multi-tenant use
|
||||
- Role-based access (admin, user, viewer) with admin bootstrap on first login
|
||||
- Emergency token allows agency staff to access app if Azure AD is down
|
||||
- Cost estimation for AI processing with `MAX_PROCESSING_COST_USD` limit
|
||||
- File retention policy: uploaded files deleted after `FILE_RETENTION_HOURS`
|
||||
|
||||
```
|
||||
┌─────────────────────┐
|
||||
│ Browser (SPA) │
|
||||
│ • MSAL auth │
|
||||
│ • Handsontable UI │
|
||||
└──────────┬──────────┘
|
||||
│ /api/, /ws (proxied)
|
||||
↓
|
||||
┌─────────────────────┐
|
||||
│ Apache 2.4 │
|
||||
│ • Reverse proxy │
|
||||
│ • Static files │
|
||||
└──────────┬──────────┘
|
||||
│
|
||||
↓
|
||||
┌─────────────────────────────────────┐
|
||||
│ Docker Container (Python/FastAPI) │
|
||||
│ • API handlers │
|
||||
│ • WebSocket server │
|
||||
│ • AI model orchestration │
|
||||
└──────────┬──────────────────────────┘
|
||||
│
|
||||
↓
|
||||
┌─────────────────────────────────────┐
|
||||
│ PostgreSQL (Docker) │
|
||||
│ • sheets, metadata, users, logs │
|
||||
│ • audit trails │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
# Local development (legacy PHP server, requires GEMINI_API_KEY in config.php)
|
||||
php -S localhost:8000
|
||||
|
||||
# Docker development (full stack)
|
||||
docker-compose up -d
|
||||
|
||||
# Stop containers
|
||||
docker-compose down
|
||||
|
||||
# View logs
|
||||
docker-compose logs -f app
|
||||
|
||||
# Access database CLI
|
||||
docker-compose exec postgres psql -U achelper -d achelper
|
||||
|
||||
# Rebuild container after code changes
|
||||
docker-compose up -d --build
|
||||
|
||||
# Run database migrations (if applicable)
|
||||
docker-compose exec app python /app/server/migrations/run.py
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `docker compose up --build`
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/ac-helper`
|
||||
|
||||
- **Server:** ai-sandbox.oliver.solutions
|
||||
- **Deploy:** `docker-compose up -d` (inside VirtualHost with Apache reverse proxy rules)
|
||||
- **URL:** https://ai-sandbox.oliver.solutions/ac-helper/
|
||||
- **Port:** 8100 (Docker app port, mapped from container's 8000)
|
||||
- **Service:** Docker Compose (no systemd service)
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/ac-helper
|
||||
|
||||
**Deployment Steps:**
|
||||
1. Copy `.env.example` → `.env` and fill in secrets
|
||||
2. Ensure Apache modules enabled: `a2enmod proxy proxy_http proxy_wstunnel`
|
||||
3. Add VirtualHost reverse proxy rules (see docker-compose.yml comments)
|
||||
4. Run `docker-compose up -d`
|
||||
5. Verify: `curl https://ai-sandbox.oliver.solutions/ac-helper/`
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Purpose |
|
||||
|----------|---------|
|
||||
| `APP_PORT` | Docker host port (default 8100) |
|
||||
| `AZURE_TENANT_ID` | Azure AD tenant for SSO |
|
||||
| `AZURE_CLIENT_ID` | Azure AD SPA application ID |
|
||||
| `AZURE_REDIRECT_URI` | Azure AD callback URL |
|
||||
| `ADMIN_EMAILS` | Comma-separated emails auto-promoted to admin on first login |
|
||||
| `EMERGENCY_TOKEN` | Long random hex string to enable bypass login (leave empty to disable) |
|
||||
| `EMERGENCY_USER_EMAIL` | Email for emergency token user |
|
||||
| `GEMINI_API_KEY` | Google Gemini API key (required for AI commands) |
|
||||
| `GOOGLE_MODEL` | Gemini model name (default: gemini-2.0-flash-exp) |
|
||||
| `OPENAI_API_KEY` | OpenAI API key |
|
||||
| `OPENAI_MODEL` | GPT model (default: gpt-4.1) |
|
||||
| `ANTHROPIC_API_KEY` | Claude API key |
|
||||
| `LLAMA_CLOUD_API_KEY` | LlamaCloud for PDF parsing |
|
||||
| `POSTGRES_PASSWORD` | PostgreSQL achelper user password |
|
||||
| `SESSION_SECRET` | Secure session signing key (generate: `python3 -c "import secrets; print(secrets.token_hex(32))"`) |
|
||||
| `MAX_UPLOAD_SIZE_MB` | Max file upload size (default: 200) |
|
||||
| `FILE_RETENTION_HOURS` | Auto-delete uploaded files after N hours |
|
||||
| `DEV_MODE` | Boolean; if true, uses DEV_USER_ID/DEV_USER_ROLE instead of Azure AD |
|
||||
| `MAX_PROCESSING_COST_USD` | Kill-switch for AI processing (prevents runaway API costs) |
|
||||
|
||||
## API / Endpoints
|
||||
|
||||
**Authentication:**
|
||||
- `POST /api/auth/login` — Exchange Azure token for session
|
||||
- `POST /api/auth/emergency-token` — Bypass login with `EMERGENCY_TOKEN`
|
||||
- `POST /api/auth/logout` — Terminate session
|
||||
|
||||
**Sheets:**
|
||||
- `GET /api/sheets/` — List all sheets for current user
|
||||
- `GET /api/sheets/{sheet_id}` — Fetch sheet data + hierarchy
|
||||
- `POST /api/sheets/` — Create new sheet
|
||||
- `PUT /api/sheets/{sheet_id}` — Update sheet metadata
|
||||
- `DELETE /api/sheets/{sheet_id}` — Delete sheet
|
||||
|
||||
**Data:**
|
||||
- `POST /api/sheets/{sheet_id}/rows` — Add/update rows
|
||||
- `DELETE /api/sheets/{sheet_id}/rows/{row_id}` — Delete row
|
||||
- `GET /api/sheets/{sheet_id}/export?format=csv` — Export to CSV
|
||||
|
||||
**AI:**
|
||||
- `POST /api/ai/command` — Execute natural language command (Gemini)
|
||||
- `WS /ws` — WebSocket for real-time updates + streaming AI responses
|
||||
|
||||
**Admin:**
|
||||
- `GET /api/admin/users` — List users (admin only)
|
||||
- `PUT /api/admin/users/{user_id}/role` — Assign role (admin only)
|
||||
- `GET /api/admin/audit-log` — View audit trail (admin only)
|
||||
|
||||
## Known Issues
|
||||
|
||||
- **Migration in progress:** Project transitioning from PHP file-based storage to PostgreSQL; some legacy PHP code paths may still exist
|
||||
- **Handsontable rendering:** Requires ResizeObserver workaround for pixel height calculation on empty grids
|
||||
- **AI model drift:** Recent commits show model swaps (gemini-3.1-pro-preview → gemini-2.0-flash-exp); verify `GOOGLE_MODEL` matches deployed version
|
||||
- **Azure AD token expiry:** Frontend auto-refreshes tokens, but edge cases with long-running sessions may require page reload
|
||||
- **WebSocket reliability:** No explicit reconnection logic documented; connection drops require manual page refresh
|
||||
- **CSV export templates:** Per-client/per-user overrides exist but global default behavior under-documented
|
||||
|
||||
## Git
|
||||
|
||||
- **Remote:** git@bitbucket.org:zlalani/ac-helper.git
|
||||
- **Recent activity:** Active development; last 20 commits show migration to PostgreSQL, Azure AD auth hardening, multi-client support, and AI model tuning
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
|
|
@ -36,4 +201,4 @@ Marketing deliverables management with AI automation. Natural language commands
|
|||
|
||||
## Related
|
||||
- [[ac-tool/AC Tool]]
|
||||
- [[gmal-scope-builder/GMAL Scope Builder]]
|
||||
- [[gmal-scope-builder/GMAL Scope Builder]]
|
||||
|
|
@ -1,22 +1,167 @@
|
|||
---
|
||||
name: "AC Tool"
|
||||
client: Oliver Internal
|
||||
client: Oliver Agency
|
||||
status: active
|
||||
tech: [Docker]
|
||||
tech: [Python, FastAPI, React, TypeScript, Docker, Azure AD, Handsontable, Claude/Gemini/GPT-4]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/ac-tool
|
||||
deploy: docker compose up --build
|
||||
url:
|
||||
deploy: docker-compose up -d
|
||||
url: https://ai-sandbox.oliver.solutions/ac-helper/
|
||||
tags: [oliver, activation-calendar]
|
||||
created: 2026-04-14
|
||||
server: ai-sandbox.oliver.solutions
|
||||
port: 8100
|
||||
---
|
||||
|
||||
## Overview
|
||||
Activation Calendar tool. Docker-based. No README — see [[ac-helper/AC Helper]] for context.
|
||||
**ac-tool** (formerly ac-helper) is a web application for AI-powered brief extraction and spreadsheet manipulation. It integrates multiple LLM providers (OpenAI, Google Gemini, Anthropic Claude) to process documents, extract key information, and populate structured data in interactive spreadsheets. Users authenticate via Azure AD (MSAL) and can manage AI jobs with real-time progress tracking via WebSocket. The tool is designed for agency workflows requiring intelligent document analysis and data consolidation.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** React (TypeScript), Handsontable (dark theme spreadsheet UI), MSAL for Azure AD auth, Axios for HTTP
|
||||
- **Backend:** Python 3 + FastAPI, uvicorn ASGI server, WebSocket support for real-time job updates
|
||||
- **Database:** File-based storage in `/app/data` (no traditional DB)
|
||||
- **Infrastructure:** Docker Compose (containerized), Apache reverse proxy frontend (static files + proxying)
|
||||
- **AI/ML:** OpenAI (gpt-4.1), Google Gemini (gemini-2.0-flash), Anthropic Claude (Opus/Sonnet), LlamaCloud (PDF parsing)
|
||||
- **Key libraries:** pydantic (config), FastAPI (REST/WS), MSAL Python (auth), llama-cloud (document extraction), google-generativeai, anthropic client
|
||||
|
||||
## Architecture
|
||||
The application follows a **three-tier proxy pattern**:
|
||||
|
||||
1. **Apache Reverse Proxy** (frontend, static files)
|
||||
- Serves frontend `/ac-helper/` from disk
|
||||
- Routes API calls (`/ac-helper/api/*`) to Docker container on `http://localhost:8100/api/`
|
||||
- Routes WebSocket (`/ac-helper/ws`) to container on `ws://localhost:8100/ws`
|
||||
- Falls back to `index.html` for SPA routing
|
||||
|
||||
2. **FastAPI Backend** (Docker container, port 8000 internally, 8100 externally)
|
||||
- REST API endpoints (`/api/*`) for auth, jobs, spreadsheets, models
|
||||
- WebSocket endpoint (`/ws`) for real-time job progress streaming
|
||||
- LLM orchestration: multi-model parallel requests, cost estimation, retry logic
|
||||
- File storage in `/app/data` (mounted volume)
|
||||
|
||||
3. **React Frontend** (SPA)
|
||||
- Azure AD login via MSAL (PKCE flow, no client secret)
|
||||
- Spreadsheet UI via Handsontable with dynamic dropdowns
|
||||
- Real-time job monitoring via WebSocket
|
||||
- Document upload, brief extraction, model selection
|
||||
|
||||
**Key Data Flow:**
|
||||
```
|
||||
User Login (Azure AD)
|
||||
→ Fetch user role/permissions
|
||||
→ Upload brief/spreadsheet
|
||||
→ Select AI models
|
||||
→ Backend: parse → extract via LLM(s) → consolidate
|
||||
→ WS: stream job progress to frontend
|
||||
→ Backend: save results to /app/data
|
||||
→ Frontend: populate Handsontable with extracted data
|
||||
```
|
||||
|
||||
## Dev Commands
|
||||
```bash
|
||||
# Local development (requires Python 3.11+, Node.js 18+)
|
||||
# Backend only (if you have server/config_runtime.py set up):
|
||||
cd /Users/ai_leed/Documents/Projects/Oliver/ac-tool
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
python server/main.py
|
||||
|
||||
# Full Docker Compose (recommended):
|
||||
docker-compose up --build
|
||||
|
||||
# Rebuild container (no cache):
|
||||
docker-compose build --no-cache && docker-compose up -d
|
||||
|
||||
# View logs:
|
||||
docker-compose logs -f app
|
||||
|
||||
# Stop container:
|
||||
docker-compose down
|
||||
|
||||
# Access locally:
|
||||
# API: http://localhost:8100/api/
|
||||
# WebSocket: ws://localhost:8100/ws
|
||||
# Frontend (static): served by Apache at https://ai-sandbox.oliver.solutions/ac-helper/
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `docker compose up --build`
|
||||
- **Server:** ai-sandbox.oliver.solutions
|
||||
- **Deploy:** `docker-compose up -d` (after pushing to Bitbucket and pulling on target server)
|
||||
- **URL:** `https://ai-sandbox.oliver.solutions/ac-helper/`
|
||||
- **Port:** 8100 (host) → 8000 (container internal)
|
||||
- **Service:** Docker Compose (container name: `ac-tool`, restart policy: `unless-stopped`)
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/ac-tool`
|
||||
|
||||
**Pre-deployment checklist:**
|
||||
1. Copy `.env.example` to `.env` and fill in all secrets (API keys, Azure credentials, SESSION_SECRET)
|
||||
2. Set `DEV_MODE=false` in production `.env`
|
||||
3. Ensure `AZURE_REDIRECT_URI` matches production domain
|
||||
4. Configure Apache VirtualHost with proxy directives (see docker-compose.yml comments)
|
||||
5. Run `docker-compose up -d` on target server
|
||||
6. Verify `/api/health` or similar endpoint responds
|
||||
|
||||
## Environment Variables
|
||||
Core environment variables (see `.env.example` for full list):
|
||||
|
||||
**Application:**
|
||||
- `APP_PORT` — Host port Docker binds to (default: 8100)
|
||||
- `DEV_MODE` — Enable mock auth, bypass real Azure AD (default: false; **must be false in production**)
|
||||
- `DEV_USER_ID` / `DEV_USER_ROLE` — Mock user credentials when DEV_MODE=true
|
||||
|
||||
**Authentication (Azure AD / MSAL):**
|
||||
- `AZURE_TENANT_ID` — Azure tenant (default: e519c2e6-bc6d-4fdf-8d9c-923c2f002385)
|
||||
- `AZURE_CLIENT_ID` — App registration client ID (default: 9079054c-9620-4757-a256-23413042f1ef)
|
||||
- `AZURE_REDIRECT_URI` — Post-login redirect (default: https://ai-sandbox.oliver.solutions/ac-helper/)
|
||||
- `MSAL_*` — Same as AZURE_* (read by backend server/config_runtime.py)
|
||||
- `SESSION_SECRET` — Session signing key (generate: `python3 -c "import secrets; print(secrets.token_hex(32))"`)
|
||||
- `ADMIN_EMAIL` — First login with this email gets admin role
|
||||
|
||||
**AI Model APIs:**
|
||||
- `OPENAI_API_KEY` / `OPENAI_MODEL` / `OPENAI_REASONING_EFFORT` / `OPENAI_TIMEOUT` / `OPENAI_MAX_RETRIES`
|
||||
- `GEMINI_API_KEY` / `GOOGLE_MODEL` / `GOOGLE_TEMPERATURE` / `GOOGLE_MAX_OUTPUT_TOKENS` / `GOOGLE_THINKING_BUDGET` / `GOOGLE_TIMEOUT`
|
||||
- `ANTHROPIC_API_KEY` / `ANTHROPIC_MODEL_OPUS` / `ANTHROPIC_MODEL_SONNET` / `ANTHROPIC_TEMPERATURE` / `ANTHROPIC_MAX_TOKENS` / `ANTHROPIC_THINKING_BUDGET` / `ANTHROPIC_TIMEOUT`
|
||||
- `LLAMA_CLOUD_API_KEY` — For advanced PDF parsing (fallback: local extraction if absent)
|
||||
|
||||
**Brief Extraction Processing:**
|
||||
- `DEFAULT_PRIMARY_MODELS` — Comma-separated model IDs for initial extraction (e.g., `anthropic-sonnet45,google-gemini20`)
|
||||
- `DEFAULT_CONSOLIDATION_MODEL` — Model for consolidating multiple extractions
|
||||
- `MINIMUM_SUCCESS_THRESHOLD` — Min successful extractions required
|
||||
- `ENABLE_COST_ESTIMATION` — Pre-calculate API costs (default: true)
|
||||
- `MAX_PROCESSING_COST_USD` — Reject jobs exceeding this cost (default: 10.00)
|
||||
- `MAX_CONCURRENT_JOBS` — Max parallel jobs (default: 5)
|
||||
|
||||
**Network & Security:**
|
||||
- `ALLOWED_ORIGINS` — CORS allowed origins (default: https://ai-sandbox.oliver.solutions)
|
||||
- `FILE_RETENTION_HOURS` — Auto-delete uploaded files after N hours (default: 24)
|
||||
- `MAX_UPLOAD_SIZE_MB` — Max file upload size (default: 200)
|
||||
- `WS_PING_INTERVAL_SECONDS` — WebSocket keepalive interval (default: 30)
|
||||
|
||||
## API / Endpoints
|
||||
**Key REST endpoints** (all prefixed `/api/`):
|
||||
- `POST /auth/token` — Exchange Azure AD token for session
|
||||
- `GET /me` — Fetch current user profile & permissions
|
||||
- `POST /jobs` — Create a new AI extraction job
|
||||
- `GET /jobs/{job_id}` — Get job status & results
|
||||
- `POST /upload` — Upload document for processing
|
||||
- `GET /spreadsheets/{sheet_id}` — Fetch spreadsheet data
|
||||
- `PUT /spreadsheets/{sheet_id}` — Update spreadsheet cells
|
||||
|
||||
**WebSocket:**
|
||||
- `GET /ws` — Connect for real-time job progress updates (emits JSON messages with job state, progress %, LLM responses)
|
||||
|
||||
## Known Issues
|
||||
- **Handsontable rendering:** Dynamic column/row updates sometimes require manual sheet reload (workaround: ResizeObserver implemented, see commit 076675f)
|
||||
- **Azure token expiry:** Auto-refresh token before every request (fixed in commit fb26d9a, but monitor for edge cases)
|
||||
- **Model config alignment:** Verify env var names match backend expectations (recent fixes in commits d71a044 & ba9af5f)
|
||||
- **PDF extraction fallback:** LlamaCloud parsing requires valid API key; local fallback used if absent (commit fc430cc)
|
||||
- **Dark theme contrast:** Recently improved (commit 45c6b2e); report accessibility issues if found
|
||||
- **No persistent database:** All data stored in `/app/data` volumes; ensure backups are configured for production
|
||||
|
||||
## Git
|
||||
- **Remote:** `git@bitbucket.org:zlalani/ac-helper.git`
|
||||
- **Default branch:** main (inferred)
|
||||
- **Recent work:** UI fixes (dark theme,
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
**Done:** Added to Obsidian second brain.
|
||||
|
|
@ -28,4 +173,4 @@ Activation Calendar tool. Docker-based. No README — see [[ac-helper/AC Helper]
|
|||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
|
||||
## Related
|
||||
- [[ac-helper/AC Helper]]
|
||||
- [[ac-helper/AC Helper]]
|
||||
|
|
@ -1,27 +1,172 @@
|
|||
---
|
||||
name: "Amazon Transcreation"
|
||||
client: "TBD"
|
||||
client: Amazon
|
||||
status: active
|
||||
server: optical-dev
|
||||
tech: []
|
||||
server: local
|
||||
tech: [Next.js 14, FastAPI, Python, PostgreSQL, Redis, Celery, Claude LLM]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/amazon-transcreation
|
||||
deploy:
|
||||
deploy: docker compose up -d
|
||||
url:
|
||||
tags:
|
||||
- project
|
||||
created: 2026-04-15
|
||||
port: 8040
|
||||
db: PostgreSQL
|
||||
---
|
||||
|
||||
## Overview
|
||||
> New project — fill in during first session.
|
||||
Amazon AI Transcreation Platform is an AI-powered web application that adapts Amazon marketing copy across 12 European locales using Claude LLM agents. It replaces a manual LibreChat workflow with a structured, one-click multi-locale processing system featuring real-time monitoring, in-app review capabilities, and proper job/file management. The platform orchestrates transcreation jobs through a Celery task queue, providing a dashboard for users to submit copy, track progress, review agent-generated translations, and manage translation memory (TM) and reference libraries.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:**
|
||||
- **Backend:**
|
||||
- **Infrastructure:**
|
||||
- **Frontend:** Next.js 14, React, TypeScript, Tailwind CSS
|
||||
- **Backend:** FastAPI (Python), Pydantic, SQLAlchemy (async), Alembic migrations
|
||||
- **Database:** PostgreSQL 16 (primary), Redis 7 (task queue & caching)
|
||||
- **Infrastructure:** Docker Compose, Celery (4 concurrent workers)
|
||||
- **AI/ML:** Anthropic Claude (default: `claude-sonnet-4-6`), LLM agent pipeline with multi-stage processing
|
||||
- **Key libraries:** python-multipart, httpx, jose (JWT auth), Azure AD MSAL (SSO)
|
||||
|
||||
## Architecture
|
||||
|
||||
The platform follows a **task-queue-driven microservices pattern**:
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ Next.js 14 │ (http://localhost:3000)
|
||||
│ (Dashboard UI) │
|
||||
└────────┬────────┘
|
||||
│ HTTP REST
|
||||
│ (3-sec polling)
|
||||
▼
|
||||
┌──────────────────────────────┐
|
||||
│ FastAPI Backend │ (http://localhost:8040)
|
||||
│ ┌──────────────────────────┐│
|
||||
│ │ Auth / Jobs / Output API ││
|
||||
│ └────────────┬─────────────┘│
|
||||
└───────────────┼──────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────┐
|
||||
│ Redis Queue │
|
||||
│ (Celery) │
|
||||
└───────┬───────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ Celery Workers (×4) │
|
||||
│ │
|
||||
│ VALIDATE ──► │
|
||||
│ SINGLE_AGENT ──► │
|
||||
│ FORMAT ──► │
|
||||
│ DONE │
|
||||
└─────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────┐
|
||||
│ PostgreSQL │
|
||||
│ (Job state) │
|
||||
└───────────────┘
|
||||
```
|
||||
|
||||
**Key Design Decisions:**
|
||||
- **Single LLM Agent Pipeline:** Replaces multi-agent flow with one optimized Claude call using full V25 prompt (validation, transcreation, formatting in one go)
|
||||
- **Async-first Backend:** Uses `asyncpg` for non-blocking database calls
|
||||
- **Job/File Management:** Each transcreation request creates a job with associated input/output files stored on disk (`/storage`)
|
||||
- **Real-time Monitoring:** Frontend polls every 3 seconds for job status updates
|
||||
- **Auth:** JWT tokens with Azure AD SSO support (optional)
|
||||
- **Locale Support:** 12 European locales (mapped to Amazon sales channels) + multi-channel TM routing
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
# Start all services (DB, Redis, Backend, Celery worker)
|
||||
make up
|
||||
|
||||
# Stop all services
|
||||
make down
|
||||
|
||||
# Rebuild containers
|
||||
make build
|
||||
|
||||
# Run database migrations
|
||||
make migrate
|
||||
|
||||
# Seed default client & test users
|
||||
make seed
|
||||
|
||||
# Run tests
|
||||
make test
|
||||
|
||||
# View logs (all services)
|
||||
make logs
|
||||
|
||||
# Access backend shell
|
||||
make shell
|
||||
|
||||
# Direct DB access
|
||||
make db-shell
|
||||
|
||||
# Direct Redis CLI
|
||||
make redis-cli
|
||||
|
||||
# Restart backend & worker (after code changes)
|
||||
make restart
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/amazon-transcreation`
|
||||
- **Server:** Local development only (docker-compose)
|
||||
- **Deploy:** `docker compose up -d`
|
||||
- **URL:** http://localhost:3000 (frontend), http://localhost:8040 (backend)
|
||||
- **Port:** 8040 (FastAPI), 3000 (Next.js frontend)
|
||||
- **Service:** Docker Compose (no systemd service configured)
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/amazon-transcreation
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Key configuration in `.env.example`:
|
||||
|
||||
- `DATABASE_URL` — PostgreSQL connection string (async driver required: `+asyncpg`)
|
||||
- `REDIS_URL` — Redis connection for Celery task queue
|
||||
- `ANTHROPIC_API_KEY` — Claude API key (required; format: `sk-ant-...`)
|
||||
- `JWT_SECRET_KEY` — Secret for signing JWT tokens (change in production)
|
||||
- `JWT_ALGORITHM` — Token algorithm (default: `HS256`)
|
||||
- `JWT_EXPIRY_HOURS` — Token expiry window (default: 8 hours)
|
||||
- `STORAGE_ROOT` — Path to job file storage (default: `/storage`)
|
||||
- `LLM_MODEL` — Claude model version (default: `claude-sonnet-4-6`)
|
||||
- `AZURE_AD_TENANT_ID` — Azure AD tenant (optional for SSO)
|
||||
- `AZURE_AD_CLIENT_ID` — Azure AD client ID (optional for SSO)
|
||||
- `AZURE_AD_SSO_ENABLED` — Enable/disable Azure AD authentication (default: `false`)
|
||||
|
||||
## API / Endpoints
|
||||
|
||||
Key routes (all on `http://localhost:8040`):
|
||||
|
||||
- `POST /api/auth/login` — User login (JWT)
|
||||
- `POST /api/jobs` — Create transcreation job
|
||||
- `GET /api/jobs/{job_id}` — Fetch job status & progress
|
||||
- `GET /api/jobs/{job_id}/output` — Retrieve completed output file
|
||||
- `POST /api/jobs/{job_id}/rerun_locale` — Re-run a failed locale
|
||||
- `GET /api/tm-registry` — List translation memory entries
|
||||
- `POST /api/tm-registry/upload` — Upload TM file
|
||||
- `GET /api/reference-library` — List reference documents
|
||||
- `POST /api/users` — Create/manage users (admin)
|
||||
|
||||
(Full OpenAPI docs at `GET /docs`)
|
||||
|
||||
## Known Issues
|
||||
|
||||
- **ORM State:** Recent fix (commit 710d931) resolves 500 errors on user updates by refreshing ORM object after database flush
|
||||
- **Dialog Warnings:** Fixed in latest commit; ensure frontend re-renders after async operations
|
||||
- **Markdown Parser:** Fixed (commit d5fa4e4) to preserve backtranslations and rationales in TM output; verify model selection doesn't break parsing
|
||||
- **TM Type Mismatch:** Fixed (commit 2c7677b) to accept both list and dict for `tm_entries_cited`
|
||||
- **Azure AD SSO:** Optional feature; only enable if tenant/client IDs configured
|
||||
- **Concurrency:** Limited to 4 Celery workers; scale up in `docker-compose.yml` if bottleneck detected
|
||||
|
||||
## Git
|
||||
|
||||
- **Remote:** `git@bitbucket.org:zlalani/amazon-transcreation.git`
|
||||
- **Branch:** (check current with `git branch`)
|
||||
- **Latest commit:** `710d931` — Fix 500 on user update: refresh ORM object after flush, fix dialog warnings
|
||||
- **Deployment branch:** (check with team; likely `main` or `develop`)
|
||||
|
||||
## Sessions
|
||||
### 2026-04-16 – Fix 500 error when updating users
|
||||
|
|
@ -79,4 +224,4 @@ created: 2026-04-15
|
|||
| 2026-04-15 | SSO Azure AD setup | Added AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_REDIRECT_URI, updated deploy script, added setup summary | docker-compose.yml, Dockerfile, deploy.sh |
|
||||
| 2026-04-15 | SSO/MSAL setup | Azure tenant/client ID configuration, removed storeAuthStateInCookie, token exchange flow | MSAL config file, authentication module |
|
||||
|
||||
## Related
|
||||
## Related
|
||||
|
|
@ -3,29 +3,128 @@ name: "Build-A-Squad"
|
|||
client: Oliver Internal
|
||||
status: active
|
||||
server: optical-web-1
|
||||
tech: [React, TypeScript, Vite, Google Gemini]
|
||||
tech: [React 19, TypeScript, Vite, Tailwind CSS, Google Gemini, Lucide React]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/build-a-squad
|
||||
deploy: npm run dev
|
||||
deploy: deploy.sh script
|
||||
url:
|
||||
tags: [oliver, staffing, fte, calculator, gemini]
|
||||
created: 2026-04-14
|
||||
port: 3000
|
||||
---
|
||||
|
||||
## Overview
|
||||
Client-side staffing calculator. Paste brief → Gemini analyzes → suggests 30+ creative assets from catalog → instant FTE breakdown by discipline and role. No backend needed.
|
||||
|
||||
3-tab workflow: **Scoping** → **Configurator** → **Squad Projection**
|
||||
**Build-A-Squad** is a client-side staffing calculator for creative agencies that helps project managers instantly estimate team composition and FTE requirements. Users paste a project brief (or upload a screenshot), the app uses Google Gemini AI to match relevant creative assets from a catalog, and generates a detailed FTE breakdown by discipline and role. Built entirely in React with no backend — all calculations and AI analysis happen in the browser.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** React 19 + TypeScript + Vite
|
||||
- **AI:** Google Gemini
|
||||
- **Infrastructure:** No Docker
|
||||
|
||||
- **Frontend:** React 19, TypeScript 5.8, Vite 6
|
||||
- **Backend:** None (client-side only)
|
||||
- **Database:** None (JSON data files in `data/`)
|
||||
- **Infrastructure:** Vite dev server (port 3000), static bundle output to `dist/`
|
||||
- **AI/ML:** Google Gemini 2.0 Flash via `@google/genai` SDK
|
||||
- **Key libraries:** Lucide React (icons), Tailwind CSS (CDN-hosted styling), @azure/msal-react (SSO support)
|
||||
|
||||
## Architecture
|
||||
|
||||
**Build-A-Squad** follows a monolithic React component architecture with three core pages (tabs) and persistent client-side state.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ App.tsx (monolithic) │
|
||||
├──────────┬──────────────┬──────────────────┤
|
||||
│ Scoping │ Configurator │ Squad Projection │
|
||||
│ Tab │ Tab │ Tab │
|
||||
├──────────┴──────────────┴──────────────────┤
|
||||
│ React Hooks (useState, useMemo) │
|
||||
│ State: selected assets, volumes, FTE │
|
||||
├──────────────────────────────────────────┤
|
||||
│ Gemini AI Integration │
|
||||
│ • Scope analysis (brief → asset matches) │
|
||||
│ • Image OCR (screenshot → text) │
|
||||
│ • Operational audit (squad → insights) │
|
||||
├──────────────────────────────────────────┤
|
||||
│ Data Layer (mockData.ts) │
|
||||
│ ├─ ASSETS_DATA (30+ creative assets) │
|
||||
│ ├─ STAFFING_ROUTES (asset → roles/hours) │
|
||||
│ ├─ ROLE_DISCIPLINE_MAP (40+ roles) │
|
||||
│ └─ Derived: ROLES, CATEGORIES, DISCIPLINES│
|
||||
└──────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Key Components & Data Flow:**
|
||||
|
||||
1. **App.tsx** — Single monolithic component managing all UI, state, and business logic. No child components; all rendering conditional on active tab.
|
||||
2. **State Management** — Pure React hooks (`useState`, `useMemo`). State includes:
|
||||
- `selectedAssets[]` — array of assets with volume (quantity)
|
||||
- `manualStaff[]` — user-added custom roles/FTE overrides
|
||||
- `scenarios[]` — named snapshots for comparison
|
||||
- `activeTab` — which tab is visible
|
||||
3. **FTE Calculation Engine** — Core algorithm in App.tsx:
|
||||
- Each asset maps to roles via `STAFFING_ROUTES`
|
||||
- Hours = base hours × asset volume
|
||||
- Total hours per role = sum of all matched assets
|
||||
- FTE = total hours ÷ billable target (default 1600/year)
|
||||
- Grouped by discipline (8 total) and role (40+)
|
||||
4. **AI Integration** — Three Gemini API calls using structured JSON schema output:
|
||||
- **Scope analysis:** Analyzes brief text, returns `{ assetIds: [...], confidence: [...] }`
|
||||
- **Image OCR:** Extracts text from uploaded images
|
||||
- **Operational audit:** Takes current squad, returns strategic insights
|
||||
5. **Data Source** — `data/*.json` files (generated by `scripts/parse-assets.ts` from `documents/`):
|
||||
- `assets.json` — 30+ asset definitions (name, category, master/adapt type, Pencil Pro flag)
|
||||
- `staffingRoutes.json` — Asset ID → role/hour mappings
|
||||
- `roleDisciplineMap.json` — Role → discipline mapping
|
||||
6. **UI Layer** — Neo-brutalist design: 4px borders, 6px offset shadows, yellow (#F5C518) focus states, Public Sans font (400/700/900).
|
||||
7. **Auth** — Azure MSAL v5 integrated for SSO (MsalProvider in index.tsx).
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
npm install # Install dependencies
|
||||
npm run dev # Start Vite dev server (port 3000)
|
||||
npm run build # Production build → dist/
|
||||
npm run preview # Preview production bundle locally
|
||||
npm run parse-catalog # Re-parse asset catalog: documents/ → data/*.json
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Dev:** `npm run dev`
|
||||
- **Build:** `npm run build`
|
||||
|
||||
- **Server:** (not specified — static hosting target TBD)
|
||||
- **Deploy:** `./deploy.sh` script exists (exact behavior not documented in provided files)
|
||||
- **URL:** (not specified)
|
||||
- **Port:** 3000 (dev), static bundle for prod
|
||||
- **Service:** (no systemd service configured)
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/build-a-squad`
|
||||
|
||||
## Environment Variables
|
||||
|
||||
- `GEMINI_API_KEY` — Google Gemini API key (required for scope analysis, image OCR, operational audit features). Set in `.env.local` for dev; injected by Vite at build time.
|
||||
- `AZURE_REDIRECT_URI` — Azure SSO redirect URI for MSAL authentication (set in `.env.production`).
|
||||
|
||||
## Three-Tab Workflow
|
||||
|
||||
1. **Scoping** — Paste a project brief or upload a screenshot. Gemini analyzes content and suggests matching assets from the catalog. Users can bulk-add matched assets with configurable volume/complexity.
|
||||
2. **Configurator** — Browse full asset catalog (30+ items) with filters: category, master/adapt type, Pencil Pro flag. Manually select assets and set quantity for each.
|
||||
3. **Squad Projection** — View calculated FTE breakdown by discipline (8) and role (40+). Support for:
|
||||
- Manual FTE overrides per role
|
||||
- Bespoke staff additions (custom roles not in catalog)
|
||||
- Scenario save/load with timestamps
|
||||
- Scenario comparison (delta indicators)
|
||||
- AI operational audit (generates strategic insights)
|
||||
- CSV/PDF export
|
||||
|
||||
## Known Issues
|
||||
|
||||
- No test runner or linter configured.
|
||||
- App is monolithic (all logic in `App.tsx` — no component decomposition).
|
||||
- Asset catalog must be manually re-parsed (`npm run parse-catalog`) if source documents change.
|
||||
- Deployment target/URL not documented.
|
||||
|
||||
## Git
|
||||
|
||||
- **Remote:** `git@bitbucket.org:zlalani/build-a-squad.git`
|
||||
- **Recent work:** SSO integration (MSAL v5), Gemini model updates, scoping expansion, role delete feature, deploy script added.
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
**Done:** Added to Obsidian second brain.
|
||||
|
|
@ -38,4 +137,4 @@ Client-side staffing calculator. Paste brief → Gemini analyzes → suggests 30
|
|||
|
||||
## Related
|
||||
- [[gmal-scope-builder/GMAL Scope Builder]]
|
||||
- [[lusa-back-planner/LUSA Back Planner]]
|
||||
- [[lusa-back-planner/LUSA Back Planner]]
|
||||
|
|
@ -1,64 +1,196 @@
|
|||
---
|
||||
name: "Enterprise AI Hub Nexus"
|
||||
client: Oliver Internal
|
||||
client: OLIVER Agency
|
||||
status: production
|
||||
tech: [Next.js 14, FastAPI, Python 3.11, PostgreSQL, Redis, Qdrant, Azure AD, Docker, Celery, Firecrawl]
|
||||
tech: [Next.js 14, FastAPI, Python 3.11, PostgreSQL, Redis, Qdrant, Azure AD, Claude/GPT-4]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/enterprise-ai-hub-nexus
|
||||
deploy: docker compose up --build
|
||||
url:
|
||||
deploy: docker-compose up -d (backend); npm run build && npm start (frontend)
|
||||
url: https://ai-sandbox.oliver.solutions
|
||||
server: optical-web-1
|
||||
tags: [oliver, ai, rag, nextjs, fastapi, azure-ad, m365, qdrant, firecrawl, production]
|
||||
created: 2026-04-14
|
||||
last_commit: 2026-04-01
|
||||
commits: 216
|
||||
port: 8000
|
||||
db: PostgreSQL 16
|
||||
---
|
||||
|
||||
## Overview
|
||||
Secure AI platform for OLIVER Agency — knowledge management, RAG chat, and Microsoft 365 productivity. Azure AD PKCE authentication. Production status.
|
||||
|
||||
Key features:
|
||||
- **RAG Chat** — vector search over uploaded knowledge base (Qdrant)
|
||||
- **M365 Integration** — SharePoint sync, To-Do tasks, email (delegated scopes)
|
||||
- **Recursive Site Crawl** — via Firecrawl /v1/crawl
|
||||
- **AI Content Structuring** — before RAG indexing
|
||||
- **Notebook Mode** — structured note-taking with AI
|
||||
**Enterprise AI Hub Nexus** is a secure, multi-tenant AI platform built for OLIVER Agency that combines RAG (Retrieval-Augmented Generation) chat, Microsoft 365 productivity integration, and knowledge base management. Users ask natural language questions answered from company documents with source citations, read emails/calendars/SharePoint via Microsoft Graph, and admins manage document uploads with intelligent indexing. The platform runs production on GCE VM `optical-web-1` with Apache reverse proxy, supports multi-language queries, department/region scoping, and role-based access (super_admin, content_manager, user).
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** Next.js 14
|
||||
- **Backend:** FastAPI + Python 3.11 + Celery workers
|
||||
- **Database:** PostgreSQL 16 (Alembic migrations, 30+ revisions)
|
||||
- **Vector DB:** Qdrant (with orphaned vector cleanup)
|
||||
- **Cache/Queue:** Redis 7
|
||||
- **Auth:** Azure AD PKCE (delegated scopes + offline_access)
|
||||
- **Crawling:** Firecrawl /v1/crawl (recursive site crawl)
|
||||
- **Infrastructure:** Docker + docker-compose
|
||||
- **Frontend:** Next.js 14, React, TypeScript, SSE streaming responses
|
||||
- **Backend:** FastAPI, Python 3.11, Gunicorn + Uvicorn, Pydantic validation
|
||||
- **Database:** PostgreSQL 16 (primary), Redis 7 (cache/queue), Qdrant (vector DB)
|
||||
- **Infrastructure:** Docker Compose (local/staging), GCE VM + Apache (production), Google Cloud Run (async document processing)
|
||||
- **AI/ML:** OpenAI (GPT-4), Anthropic (Claude Haiku for reranking), Google Gemini, LlamaParse (PDF extraction), Firecrawl (web scraping)
|
||||
- **Auth:** Microsoft Entra ID (Azure AD) PKCE SPA flow, JWT HS256
|
||||
- **Key libraries:** `python-jose`, `fastapi-jwt-extended`, `sqlalchemy`, `celery`, `qdrant-client`, `langchain`, `microsoft-graph-core`
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Next.js 14 frontend
|
||||
↓ Azure AD PKCE auth
|
||||
FastAPI backend
|
||||
├── Celery beat (SharePoint sync, scheduled tasks)
|
||||
├── Qdrant (vector search / RAG)
|
||||
├── PostgreSQL (data + migrations)
|
||||
└── Redis (queue + cache)
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Client (Next.js 14 SPA) │
|
||||
│ (Azure AD PKCE login → JWT in localStorage) │
|
||||
└────────────────────────┬────────────────────────────────────┘
|
||||
│ HTTPS
|
||||
┌────────────────┴────────────────┐
|
||||
│ Apache Reverse Proxy │
|
||||
│ (optical-web-1) │
|
||||
└────────────────┬────────────────┘
|
||||
│
|
||||
┌────────────────┴──────────────────────────┐
|
||||
│ │
|
||||
┌───────▼────────────────────┐ ┌─────────────▼───────┐
|
||||
│ FastAPI Backend │ │ Cloud Run Async │
|
||||
│ (Docker container) │ │ (Heavy doc proc) │
|
||||
│ │ │ │
|
||||
│ • /api/v1/chat (RAG) │ │ • PDF parsing │
|
||||
│ • /api/v1/auth │ │ • Document chunking │
|
||||
│ • /api/v1/documents │ │ • Embedding gen │
|
||||
│ • /api/v1/m365 (Personal) │ │ • Vector indexing │
|
||||
│ • /api/v1/admin │ │ │
|
||||
│ │ └─────────────────────┘
|
||||
└───────┬────────────────────┘
|
||||
│
|
||||
┌───┴──────┬──────────┬──────────┐
|
||||
│ │ │ │
|
||||
┌───▼──┐ ┌────▼────┐ ┌──▼───┐ ┌────▼─────┐
|
||||
│ PG │ │ Redis │ │Qdrant│ │ uploads/ │
|
||||
│ 16 │ │ 7 │ │ VDB │ │ files │
|
||||
└──────┘ └─────────┘ └──────┘ └──────────┘
|
||||
|
||||
External APIs:
|
||||
• Microsoft Graph (emails, calendar, SharePoint, OneDrive)
|
||||
• OpenAI / Anthropic / Google
|
||||
• LlamaParse / Firecrawl (web + PDF parsing)
|
||||
```
|
||||
|
||||
## Docker Services
|
||||
- `nexus-postgres` (PostgreSQL 16)
|
||||
- `nexus-redis` (Redis 7)
|
||||
- `nexus-qdrant` (Qdrant vector DB)
|
||||
- `nexus-backend` (FastAPI)
|
||||
- frontend (Next.js 14)
|
||||
**Key Data Flow:**
|
||||
1. **RAG Chat:**
|
||||
- User query → multi-query expansion (3 variants: translated + UK/US English)
|
||||
- Parallel Qdrant vector search (60 candidates max)
|
||||
- Claude Haiku reranking (0–10 score) → top 5 chunks
|
||||
- LLM response with SSE streaming + source citations
|
||||
|
||||
2. **Document Upload:**
|
||||
- File → deduplication (SHA-256) → Cloud Run async processor
|
||||
- LlamaParse/Firecrawl extraction → LLM chunking + summarization
|
||||
- Chunk embedding + storage in Qdrant
|
||||
- Metadata (title, department, region, SharePoint link) in PostgreSQL
|
||||
|
||||
3. **Personal Assistant (M365):**
|
||||
- User consent → delegated Microsoft Graph token
|
||||
- Agentic loop (up to 5 rounds) calling tools in parallel:
|
||||
- Read emails, summarize threads
|
||||
- Read calendar events
|
||||
- Search OneDrive / SharePoint
|
||||
- Proactive token refresh + `needs_reconsent` flag if expired
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
# Local development (all services)
|
||||
cd /Users/ai_leed/Documents/Projects/Oliver/enterprise-ai-hub-nexus
|
||||
docker-compose up -d
|
||||
|
||||
# Backend only (if containers already running)
|
||||
cd backend
|
||||
python -m uvicorn main:app --reload --host 0.0.0.0 --port 8000
|
||||
|
||||
# Frontend (separate terminal)
|
||||
cd frontend
|
||||
npm install
|
||||
npm run dev # runs on http://localhost:3000
|
||||
|
||||
# Run migrations
|
||||
docker exec nexus-backend alembic upgrade head
|
||||
|
||||
# View logs
|
||||
docker-compose logs -f backend
|
||||
docker-compose logs -f db
|
||||
|
||||
# Shut down
|
||||
docker-compose down
|
||||
|
||||
# Production deployment (on optical-web-1)
|
||||
# See "Deployment" section below
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `docker compose up --build`
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/enterprise-ai-hub-nexus`
|
||||
- **Status:** Production
|
||||
|
||||
## Key Handover Documents
|
||||
The project has extensive context handover docs in root:
|
||||
`CONTEXT_HANDOVER_PHASE4.md`, `PHASE5.md`, `PHASE6_*.md`, etc.
|
||||
- **Server:** `optical-web-1` (GCE VM, production)
|
||||
- **Deploy:**
|
||||
- Backend: `docker-compose -f docker-compose.prod.yml up -d` (or via CI/CD)
|
||||
- Frontend: built as static bundle, served by Apache
|
||||
- Database: PostgreSQL 16 managed on VM
|
||||
- Reverse proxy: Apache 2 (SSL/TLS, CORS)
|
||||
- **URL:** https://ai-sandbox.oliver.solutions
|
||||
- **Port:** 8000 (backend), 443 (Apache HTTPS)
|
||||
- **Service:** Docker Compose (no systemd service; managed by docker-compose)
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/enterprise-ai-hub-nexus
|
||||
|
||||
**Production Notes:**
|
||||
- Apache acts as reverse proxy with SSL termination
|
||||
- Backend runs in Docker Compose with health checks
|
||||
- Qdrant, PostgreSQL, Redis persist data to volumes
|
||||
- Cloud Run microservice (`nexus-doc-processor`) handles heavy async jobs
|
||||
- Environment variables loaded from `.env` (not in version control)
|
||||
- Gunicorn workers: 4 (2 * CPU + 1 rule)
|
||||
|
||||
## Environment Variables
|
||||
|
||||
**Core:**
|
||||
- `ENVIRONMENT` — `production`, `staging`, or `development`
|
||||
- `DEBUG` — `true`/`false` (development only)
|
||||
- `BACKEND_BASE_URL` — public backend URL for CORS & webhooks (e.g., `https://ai-sandbox.oliver.solutions`)
|
||||
|
||||
**Database & Cache:**
|
||||
- `DATABASE_URL` — PostgreSQL connection string (docker host is `db`)
|
||||
- `POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD` — DB credentials
|
||||
- `REDIS_URL` — Redis connection string
|
||||
- `QDRANT_URL` — Qdrant vector DB HTTP endpoint
|
||||
|
||||
**Authentication (Azure AD):**
|
||||
- `ENTRA_CLIENT_ID` — OAuth2 app client ID
|
||||
- `ENTRA_CLIENT_SECRET` — OAuth2 app secret
|
||||
- `ENTRA_TENANT_ID` — Azure tenant ID
|
||||
- `ENTRA_REDIRECT_URI` — OAuth2 callback (e.g., `https://ai-sandbox.oliver.solutions/nexus/api/v1/auth/callback`)
|
||||
|
||||
**JWT:**
|
||||
- `JWT_SECRET` — 32+ character secret (generate: `openssl rand -hex 32`)
|
||||
- `JWT_ALGORITHM` — `HS256`
|
||||
- `JWT_EXPIRATION_MINUTES` — token lifetime in minutes (default: 15)
|
||||
- `REFRESH_TOKEN_EXPIRATION_DAYS` — refresh token lifetime (default: 7)
|
||||
|
||||
**LLM APIs:**
|
||||
- `OPENAI_API_KEY` — GPT-4 queries
|
||||
- `ANTHROPIC_API_KEY` — Claude Haiku reranking
|
||||
- `GOOGLE_API_KEY` — Gemini fallback
|
||||
- `LLAMAPARSE_API_KEY` — cloud PDF parsing (optional)
|
||||
- `FIRECRAWL_API_KEY` — web scraping
|
||||
|
||||
**Logging & Rate Limiting:**
|
||||
- `LOG_LEVEL` — `DEBUG`, `INFO`, `WARNING`, `ERROR`
|
||||
- `LOG_FORMAT` — `json` (production) or `text` (dev)
|
||||
- `RATE_LIMIT_PER_MINUTE` — requests per minute per user (default: 60)
|
||||
|
||||
**Upload:**
|
||||
- `MAX_UPLOAD_SIZE_MB` — max file size (default: 100 MB)
|
||||
|
||||
**Networking:**
|
||||
- `CORS_ORIGINS` — comma-separated allowed origins
|
||||
- `TRUSTED_HOSTS` — comma-separated trusted hosts (empty = allow all)
|
||||
- `WORKERS_COUNT` — Gunicorn worker count (default: 4)
|
||||
|
||||
## API / Endpoints
|
||||
|
||||
**Authentication:**
|
||||
- `POST /api/v1/auth/login` — initiate Azure AD PKCE login
|
||||
- `GET /api/v1/auth/callback` — OAuth2 callback (redirected from Azure AD)
|
||||
- `POST /api/
|
||||
|
||||
## Timeline / Git History
|
||||
| Date | Change |
|
||||
|
|
@ -86,4 +218,4 @@ The project has extensive context handover docs in root:
|
|||
## Related
|
||||
- [[Oliver-ai-bot_2.0/Oliver AI Bot 2.0]] (earlier version)
|
||||
- [[sandbox-notebookllamalm-nextjs/Sandbox NotebookLM]] (same server cluster)
|
||||
- [[DevOps_Click_UP_sync/DevOps ClickUp Sync]]
|
||||
- [[DevOps_Click_UP_sync/DevOps ClickUp Sync]]
|
||||
|
|
@ -2,28 +2,131 @@
|
|||
name: "Ferrero AC Booking Tool"
|
||||
client: Ferrero
|
||||
status: active
|
||||
server: optical-web-1
|
||||
tech: [Node.js, HTML/JS, Box API]
|
||||
server: local
|
||||
tech: [HTML, CSS, JavaScript, Node.js]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/ferrero-ac-creator
|
||||
deploy: node server.js
|
||||
url: http://localhost:3456
|
||||
tags: [ferrero, booking, omg, csv, box]
|
||||
created: 2026-04-14
|
||||
port: 3456
|
||||
---
|
||||
|
||||
## Overview
|
||||
Browser-based booking tool for Ferrero communication assets. Produces CSV files for OMG media booking system import. Box API integration for direct folder saves ("Send to OMG").
|
||||
|
||||
ferrero-ac-creator is a browser-based tool for generating and managing Ferrero communication asset (AC) bookings. It transforms user input into CSV files formatted for import into the OMG media booking system. The tool enables marketing teams to efficiently create, bulk-edit, and export asset booking records with computed fields (Title, Category, Media Type, etc.) derived from master filenames, asset metadata, and ISO codes. It supports both browser-based CSV download and server-side export directly to a Box folder for OMG ingestion.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** HTML/JS
|
||||
- **Backend:** Node.js (`server.js`)
|
||||
- **Integrations:** Box API
|
||||
- **Infrastructure:** No Docker
|
||||
|
||||
- **Frontend:** HTML5, CSS3, vanilla JavaScript (no frameworks or external libraries)
|
||||
- **Backend:** Node.js (stdlib only, zero npm dependencies)
|
||||
- **Database:** None
|
||||
- **Infrastructure:** Local HTTP server
|
||||
- **AI/ML:** None
|
||||
- **Key libraries:** None (single-file, zero-dependency architecture)
|
||||
|
||||
## Architecture
|
||||
|
||||
This is a **zero-dependency single-page application** with minimal backend:
|
||||
|
||||
**Frontend (`index.html`)**
|
||||
- Single-file HTML document containing all markup, CSS, and JavaScript
|
||||
- Input table with editable rows: Master File Name, Local Meta ID, Asset Type, Media Type, Aspect Ratio, ISO Code, and date pickers (Supply/Live/End dates)
|
||||
- Interactive features:
|
||||
- Row selection with checkboxes
|
||||
- Bulk edit modal for dropdown and date fields across multiple rows
|
||||
- `runAnalyse()` function that transforms input rows into output rows with computed fields
|
||||
- CSV export to browser download or via server POST
|
||||
- All dropdown data (Asset Type, Media Type, Aspect Ratio, ISO Code) embedded as JavaScript arrays
|
||||
|
||||
**Backend (`server.js`)**
|
||||
- Minimal Node.js HTTP server listening on port 3456
|
||||
- Two responsibilities:
|
||||
1. Serves `index.html` as static file (SPA host)
|
||||
2. `POST /api/send-to-omg` — receives CSV data and writes to hardcoded Box folder path (`OMG_OUTPUT_DIR`)
|
||||
|
||||
**Data Flow**
|
||||
1. User populates input table manually or via bulk edit
|
||||
2. Click "Analyse" → `runAnalyse()` executes transformation logic
|
||||
3. Title generation mirrors Excel Column J formula: parses Master File Name segments, looks up asset/media type codes, combines with aspect ratio, ISO code, Local Meta ID
|
||||
4. Output table populated with computed columns: Title, Status (always "Booked"), Category, Media, Sub Media, Destination, Format, Language, Country, Creative Execution, Quantity
|
||||
5. User exports via "Download CSV" (browser) or "Send to OMG" (server POST with campaign number)
|
||||
6. CSV written with headers: `Title,Status,Category,Media,Sub media,Destination,Format,Supply date,Live date,End date,Special instructions,Language,Country,Creative Execution,Quantity`
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ index.html (SPA) │
|
||||
│ ┌─────────────────────────────────────┐│
|
||||
│ │ Input Table (editable rows) ││
|
||||
│ │ - Master File Name, Local Meta ID ││
|
||||
│ │ - Dropdowns: Asset, Media, Ratio... ││
|
||||
│ │ - Date pickers ││
|
||||
│ └─────────────────────────────────────┘│
|
||||
│ ↓ runAnalyse() │
|
||||
│ ┌─────────────────────────────────────┐│
|
||||
│ │ Output Table (computed fields) ││
|
||||
│ │ - Title (from Excel formula logic) ││
|
||||
│ │ - Category, Media, Destination... ││
|
||||
│ └─────────────────────────────────────┘│
|
||||
│ ↓ Download CSV ↓ Send to OMG │
|
||||
└─────────────────────────────────────────┘
|
||||
↓ browser ↓ POST /api/send-to-omg
|
||||
.csv file ┌──────────────────┐
|
||||
│ server.js │
|
||||
│ Write to Box dir │
|
||||
└──────────────────┘
|
||||
```
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
# Start the development server
|
||||
node server.js
|
||||
|
||||
# Open in browser
|
||||
# http://localhost:3456
|
||||
|
||||
# Or open index.html directly (download CSV works without server)
|
||||
open index.html
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `node server.js` → http://localhost:3456
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/ferrero-ac-creator`
|
||||
- **Note:** "Download CSV" works without server (open `index.html` directly)
|
||||
|
||||
- **Server:** Local development only (no production deployment documented)
|
||||
- **Deploy:** `node server.js` (manual, no automated CI/CD)
|
||||
- **URL:** http://localhost:3456
|
||||
- **Port:** 3456
|
||||
- **Service:** None (manual process, not systemd)
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/ferrero-ac-creator
|
||||
|
||||
## Environment Variables
|
||||
|
||||
`OMG_OUTPUT_DIR` — Hardcoded in `server.js` to a specific user's Box folder path. Must be manually updated per machine for "Send to OMG" feature to work. Current value points to `/Users/pauljohns/Library/CloudStorage/Box-Box/AUTOMATION_MAIN/OMG_NIFI/OMG_CORE_AC_INGEST`
|
||||
|
||||
## API / Endpoints
|
||||
|
||||
- **`POST /api/send-to-omg`**
|
||||
- Accepts JSON payload with CSV data and 7-digit campaign number
|
||||
- Writes file as `OMG{campaign_number}_ACINGEST.csv` to `OMG_OUTPUT_DIR`
|
||||
- Returns success/error response
|
||||
- Requires valid campaign number (7 digits)
|
||||
|
||||
- **`GET /`**
|
||||
- Serves `index.html` static file
|
||||
|
||||
## Known Issues
|
||||
|
||||
- **Hardcoded Box path:** `OMG_OUTPUT_DIR` in `server.js` is hardcoded per machine; must be manually updated for each developer/environment
|
||||
- **No authentication:** Recent git commit (1638e95) adds "session-based login page and auth middleware" but details not documented — verify implementation status
|
||||
- **No npm install:** Zero dependencies by design, but limits extensibility; any future library additions require architectural changes
|
||||
- **Dropdown data maintenance:** All dropdown values embedded in `index.html` as JavaScript arrays; requires manual editing to update (originally sourced from `Example/AC_Booking.xlsx`)
|
||||
- **Excel formula parity:** Title generation mirrors Excel Column J formula; changes to formula logic must be synced manually
|
||||
|
||||
## Git
|
||||
|
||||
- **Remote:** git@bitbucket.org:zlalani/ferrero-ac-creator.git
|
||||
- **Branch:** Main/develop (not specified in documentation)
|
||||
- **Recent activity:** Auth middleware added (commit 1638e95); last commit from Bitbucket web UI suggests active maintenance
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
|
|
@ -33,4 +136,4 @@ Browser-based booking tool for Ferrero communication assets. Produces CSV files
|
|||
## Change Log
|
||||
| Date | Requested | Changed | Files |
|
||||
|------|-----------|---------|-------|
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
|
|
@ -2,66 +2,173 @@
|
|||
name: "GMAL Scope Builder"
|
||||
client: Oliver Internal
|
||||
status: active
|
||||
tech: [Python, FastAPI, React, TypeScript, Vite, PostgreSQL, Claude Opus 4.6, Docker, Azure SSO]
|
||||
tech: [React, TypeScript, FastAPI, PostgreSQL, Claude API, Docker, Vite]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/gmal-scope-builder
|
||||
deploy: docker compose up --build
|
||||
url:
|
||||
server: optical-dev
|
||||
deploy: docker compose up -d
|
||||
url: http://localhost:3010
|
||||
server: local
|
||||
tags: [oliver, gmal, ratecard, fte, ai, claude, postgres, azure-sso]
|
||||
created: 2026-04-14
|
||||
last_commit: 2026-03-31
|
||||
commits: 23
|
||||
port: 3010
|
||||
db: PostgreSQL
|
||||
---
|
||||
|
||||
## Overview
|
||||
AI-powered scoping tool matching client deliverables against GMAL master asset database. Builds ratecards with hours per role and calculates FTE headcount.
|
||||
|
||||
**Workflow:**
|
||||
1. Upload client brief (Word/Excel) → Claude Opus 4.6 parses it
|
||||
2. AI matches deliverables to GMAL asset catalog
|
||||
3. System builds ratecard (hours per role) + FTE model
|
||||
4. Excel export with GMAL standard caveats
|
||||
**GMAL Scope Builder** is a Dockerized AI-powered web application that helps teams scope new client projects by matching client deliverables (uploaded Word/Excel documents) against a standardized GMAL (Global Master Asset List) database. It uses Claude Opus 4.6 to parse documents, match assets intelligently, generate ratecards with hourly rates per role, and calculate team FTE headcount models. The tool combines human expertise with AI to accelerate scoping workflows and ensure consistency across projects.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** React 18 + TypeScript + Vite + React Router + Axios (port 3010)
|
||||
- **Backend:** FastAPI + SQLAlchemy async + asyncpg + Uvicorn (port 8001)
|
||||
- **Database:** PostgreSQL 16 (port 5433)
|
||||
- **AI:** Claude Opus 4.6 via Anthropic SDK (tool_use for structured output)
|
||||
- **Auth:** Azure SSO (MSAL v5) + `DEV_AUTH_BYPASS` env var for local dev
|
||||
- **Infrastructure:** Docker + docker-compose (3 services)
|
||||
|
||||
## Ports
|
||||
| Service | Port |
|
||||
|---------|------|
|
||||
| Frontend (Vite) | 3010 |
|
||||
| Backend API | 8001 |
|
||||
| PostgreSQL | 5433 |
|
||||
- **Frontend:** React 18 + TypeScript + Vite + React Router + Axios
|
||||
- **Backend:** FastAPI + SQLAlchemy (async) + asyncpg + Uvicorn
|
||||
- **Database:** PostgreSQL 16 (async queries with asyncpg)
|
||||
- **Infrastructure:** Docker + docker-compose (3 services: db, backend, frontend)
|
||||
- **AI/ML:** Claude Opus 4.6 via Anthropic SDK (tool_use for structured output, per-project token tracking)
|
||||
- **Document Parsing:** openpyxl (Excel), python-docx (Word)
|
||||
- **Authentication:** Azure SSO (MSAL v5) with dev bypass option
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ React Frontend (port 3010) │
|
||||
│ ┌─ Dashboard ─ NewProject ─ ProjectView ─ GmalBrowser ─┐
|
||||
│ └────────────── Debug Panel + AI Cost Tracker ──────────┘
|
||||
└────────────────────────┬────────────────────────────────┘
|
||||
│ Axios + API Client
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ FastAPI Backend (port 8001/8002) │
|
||||
│ ┌─ Routes ───────────────────────────────────────────┐ │
|
||||
│ │ /api/gmal/* ─ GMAL catalog endpoints │ │
|
||||
│ │ /api/ingest ─ Parse & load Excel file │ │
|
||||
│ │ /api/projects/* ─ Project CRUD + workflow │ │
|
||||
│ │ /api/matching/* ─ AI document matching │ │
|
||||
│ │ /api/ratecard/* ─ Build ratecards & exports │ │
|
||||
│ │ /api/health ─ Healthcheck │ │
|
||||
│ └─ Services ────────────────────────────────────────┘ │
|
||||
│ │ claude_client.py ─ AI calls + token tracking │ │
|
||||
│ │ gmal_service.py ─ GMAL queries & caching │ │
|
||||
│ │ project_service.py ─ Project state management │ │
|
||||
│ │ matching_service.py ─ Asset matching logic │ │
|
||||
│ │ ratecard_service.py ─ Ratecard building & export │ │
|
||||
│ └─ Models (SQLAlchemy ORM) ────────────────────────┘ │
|
||||
│ │ gmal.py ─ GmalAsset, Role, ServiceLine │ │
|
||||
│ │ project.py ─ Project, ClientAsset, Match │ │
|
||||
│ └────────────────────────────────────────────────────┘
|
||||
└────────────────────────┬────────────────────────────────┘
|
||||
│ asyncpg
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ PostgreSQL 16 (port 5433) │
|
||||
│ ┌─ gmal_* tables (390 assets, categories) │
|
||||
│ ├─ projects (scope definitions) │
|
||||
│ ├─ client_assets (extracted deliverables) │
|
||||
│ ├─ matches (AI-matched assets + confidence) │
|
||||
│ └─ ratecard_lines (computed hours/rates) │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Data Flow:**
|
||||
1. User uploads client brief (Word/Excel) → Backend extracts deliverables via Claude
|
||||
2. Extracted items matched against GMAL catalog → Claude ranks by confidence + rationale
|
||||
3. Matched assets → Ratecard builder computes hours/role based on GMAL templates
|
||||
4. Ratecard + FTE model exported as Excel with caveats
|
||||
5. All AI calls tracked: per-project tokens, global cost, 50-call debug log
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
# Docker (primary)
|
||||
cp .env.example .env # Add ANTHROPIC_API_KEY
|
||||
docker compose build
|
||||
# === Docker (primary workflow) ===
|
||||
docker compose build # Build all images
|
||||
docker compose up -d # Start services in background
|
||||
docker compose logs backend --tail 50 # View backend logs
|
||||
docker compose logs frontend --tail 20 # View frontend logs
|
||||
docker compose down # Stop all services
|
||||
|
||||
# === One-time setup (local dev) ===
|
||||
cp .env.example .env # Create env file
|
||||
# Edit .env and add ANTHROPIC_API_KEY
|
||||
# Place "U-Studio GMAL Asset Job Routes Apr25 ForFranky.xlsx" in data/
|
||||
docker compose up -d
|
||||
docker compose logs backend --tail 50
|
||||
curl -X POST http://localhost:8001/api/gmal/ingest # Populate GMAL from Excel
|
||||
|
||||
# One-time: populate GMAL catalog (place Excel in data/)
|
||||
curl -X POST http://localhost:8001/api/gmal/ingest
|
||||
# === Frontend only (without Docker) ===
|
||||
cd frontend
|
||||
npm install
|
||||
npm run dev # Vite dev server with HMR (port 5173)
|
||||
npm run build # TypeScript compile + Vite bundle
|
||||
|
||||
# DB backup
|
||||
docker compose exec db pg_dump -U scope_user -d scope_builder > backups/dump.sql
|
||||
# === Backend only (without Docker) ===
|
||||
cd backend
|
||||
pip install -r requirements.txt
|
||||
uvicorn app.main:app --reload --port 8000
|
||||
|
||||
# Without Docker
|
||||
cd frontend && npm install && npm run dev
|
||||
cd backend && uvicorn app.main:app --reload --port 8000
|
||||
# === Database operations ===
|
||||
# Backup
|
||||
docker compose exec db pg_dump -U scope_user -d scope_builder > backups/scope_builder_$(date +%Y%m%d).sql
|
||||
|
||||
# Restore
|
||||
docker compose exec -T db psql -U scope_user -d scope_builder < backups/scope_builder_deploy.sql
|
||||
|
||||
# Check GMAL data loaded
|
||||
docker compose exec db psql -U scope_user -d scope_builder -c "SELECT COUNT(*) FROM gmal_assets;"
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `docker compose up --build`
|
||||
- **Deploy script:** `deploy.sh`
|
||||
- **Base path:** `/gsb`
|
||||
|
||||
- **Server:** local (development) or custom server for production
|
||||
- **Deploy:** `docker compose build && docker compose up -d` (with .env configured)
|
||||
- **URL (dev):** http://localhost:3010
|
||||
- **URL (prod):** Configure in `docker-compose.yml` and Azure SSO redirect URIs
|
||||
- **Port (frontend):** 3010 (mapped from container port 3000)
|
||||
- **Port (backend):** 8001 or 8002 (mapped from container port 8000)
|
||||
- **Port (database):** 5433 (mapped from container port 5432)
|
||||
- **Service:** None (Docker-managed; no systemd)
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/gmal-scope-builder`
|
||||
|
||||
**Deployment Steps (to a server):**
|
||||
1. Clone repo: `git clone git@bitbucket.org:zlalani/gmal-scope-builder.git`
|
||||
2. Copy GMAL Excel and DB backup: `scp backups/scope_builder_deploy.sql user@server:/path/to/data/` and `scp "data/U-Studio GMAL...xlsx" user@server:/path/to/data/`
|
||||
3. Create `.env` with `POSTGRES_PASSWORD`, `ANTHROPIC_API_KEY`, `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`
|
||||
4. Optionally edit `docker-compose.yml` to change exposed ports
|
||||
5. Build & start: `docker compose build && docker compose up -d`
|
||||
6. Import DB: `docker compose exec -T db psql -U scope_user -d scope_builder < backups/scope_builder_deploy.sql`
|
||||
7. Verify: `curl http://localhost:8001/api/health` and open frontend in browser
|
||||
|
||||
## Environment Variables
|
||||
|
||||
- `POSTGRES_PASSWORD` — Database password (default: `scope_pass_2024`)
|
||||
- `ANTHROPIC_API_KEY` — Anthropic SDK key for Claude Opus 4.6 calls (required for AI features)
|
||||
- `AZURE_TENANT_ID` — Azure AD tenant ID for SSO (required in production)
|
||||
- `AZURE_CLIENT_ID` — Azure AD application client ID for MSAL (required in production)
|
||||
- `DEV_AUTH_BYPASS` — Set to any value to skip Azure SSO in local dev (NEVER use in production)
|
||||
- `VITE_DEV_AUTH_BYPASS` — Frontend flag to match backend auth bypass
|
||||
- `DATA_DIR` — Absolute path to directory containing GMAL Excel file (defaults to `./data` if not set)
|
||||
- `DATABASE_URL` — Auto-set by backend container; `postgresql+asyncpg://scope_user:PASSWORD@db:5432/scope_builder`
|
||||
- `DATABASE_URL_SYNC` — Sync variant for alembic migrations and initial setup
|
||||
|
||||
## API Endpoints
|
||||
|
||||
**GMAL Catalog:**
|
||||
- `GET /api/gmal/stats` — Asset counts by category
|
||||
- `GET /api/gmal/assets` — List all GMAL assets (paginated)
|
||||
- `GET /api/gmal/assets/{asset_id}` — Get single asset
|
||||
- `GET /api/gmal/roles` — List all roles
|
||||
- `POST /api/gmal/ingest` — Parse Excel file and populate catalog
|
||||
|
||||
**Projects:**
|
||||
- `POST /api/projects` — Create new project
|
||||
- `GET /api/projects` — List projects
|
||||
- `GET /api/projects/{project_id}` — Get project details
|
||||
- `PUT /api/projects/{project_id}` — Update project
|
||||
- `DELETE /api/projects/{project_id}` — Delete project
|
||||
|
||||
**Matching & Extraction:**
|
||||
- `POST /api/projects
|
||||
|
||||
## Timeline / Git History
|
||||
| Date | Change |
|
||||
|------|--------|
|
||||
|
|
@ -87,4 +194,4 @@ cd backend && uvicorn app.main:app --reload --port 8000
|
|||
|
||||
## Related
|
||||
- [[build-a-squad/Build A Squad]] (same FTE scoping concept)
|
||||
- [[ac-helper/AC Helper]]
|
||||
- [[ac-helper/AC Helper]]
|
||||
|
|
@ -2,26 +2,141 @@
|
|||
name: "H&M O2E Tool (OMG Static)"
|
||||
client: H&M
|
||||
status: active
|
||||
server: baic (web-03)
|
||||
tech: [HTML/JS, One2Edit API]
|
||||
server: local
|
||||
tech: [JavaScript, PHP, HTML/CSS, One2Edit API, Azure AD/MSAL, REST API]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/hm-o2e-tool
|
||||
deploy: static / python -m http.server
|
||||
deploy: deploy.sh
|
||||
url:
|
||||
tags: [hm, one2edit, image-relinking, o2e]
|
||||
created: 2026-04-14
|
||||
---
|
||||
|
||||
## Overview
|
||||
One2Edit API tool for H&M documents. Automated image relinking (by File or by Folder), document exporting, image status checking.
|
||||
hm-o2e-tool is a web-based document management application for H&M that integrates with the One2Edit API to automate image relinking, document exporting, and image status validation. It provides a single-page interface for batch processing InDesign documents, checking image link status across folders, and exporting documents in multiple formats (INDD, PDF). The tool uses Azure AD SSO authentication via MSAL.js and manages One2Edit sessions internally for safe document operations.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** HTML/JS
|
||||
- **Integrations:** One2Edit API
|
||||
- **Infrastructure:** Static (no backend)
|
||||
- **Frontend:** Vanilla JavaScript, HTML5, CSS3, MSAL.js (Microsoft Authentication Library)
|
||||
- **Backend:** PHP (config.php for credentials/asset configuration)
|
||||
- **Database:** N/A (stateless; relies on One2Edit API)
|
||||
- **Infrastructure:** Static file deployment with Apache (.htaccess for routing)
|
||||
- **AI/ML:** N/A
|
||||
- **Key libraries:** MSAL.js (authentication), One2Edit REST API client
|
||||
|
||||
## Architecture
|
||||
The application follows a **single-page application (SPA)** pattern with client-side session management:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Browser (index.html) │
|
||||
│ ┌────────────────────────────────────────┐ │
|
||||
│ │ MSAL.js Authentication (Azure AD SSO) │ │
|
||||
│ └────────────────────────────────────────┘ │
|
||||
│ ┌────────────────────────────────────────┐ │
|
||||
│ │ Action Selector (5 operations) │ │
|
||||
│ │ • Relink Images (File/Folder) │ │
|
||||
│ │ • Image Status Check │ │
|
||||
│ │ • Export INDD │ │
|
||||
│ │ • Export PDF │ │
|
||||
│ │ • Ratio Check │ │
|
||||
│ └────────────────────────────────────────┘ │
|
||||
│ ┌────────────────────────────────────────┐ │
|
||||
│ │ Debug Log (AJAX request/response) │ │
|
||||
│ └────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────┘
|
||||
↓ AJAX (GET config.php)
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ config.php │
|
||||
│ • API credentials │
|
||||
│ • Hardcoded asset IDs (H&M logos) │
|
||||
│ • Campaign folder mappings │
|
||||
└─────────────────────────────────────────────┘
|
||||
↓ REST calls (One2Edit API)
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ One2Edit API │
|
||||
│ • document.session.open/save/close │
|
||||
│ • Image linking & relinking │
|
||||
│ • Document export (INDD/PDF) │
|
||||
│ • Asset library queries │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Key Design Decisions:**
|
||||
1. **Session-based operations:** All document edits wrapped in One2Edit session lifecycle (open → operate → save → close)
|
||||
2. **Image matching hierarchy:** Hardcoded logos (H&M sRGB/CMYK) → fuzzy search in campaign folder structure
|
||||
3. **Batch processing:** Folder operations use configurable delays (5s relink, 2s document, 500ms status) to avoid API throttling
|
||||
4. **Campaign folder structure:** Assets organized as `Assets/[Campaign]/Links/[RGB|CMYK]/`
|
||||
5. **CMYK detection:** Files ending `39L_TAC330.tif` → CMYK; all others → RGB
|
||||
6. **Authentication:** Azure AD SSO via MSAL; no backend user management
|
||||
|
||||
## Dev Commands
|
||||
```bash
|
||||
# Open in browser (no build step; static files)
|
||||
open /Users/ai_leed/Documents/Projects/Oliver/hm-o2e-tool/index.html
|
||||
|
||||
# Or serve locally with Python
|
||||
cd /Users/ai_leed/Documents/Projects/Oliver/hm-o2e-tool
|
||||
python3 -m http.server 8000
|
||||
# Visit http://localhost:8000
|
||||
|
||||
# Deploy to production
|
||||
./deploy.sh
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** Open `index.html` or `python -m http.server`
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/hm-o2e-tool`
|
||||
- **Server:** local (static file deployment)
|
||||
- **Deploy:** `./deploy.sh` (idempotent bash script)
|
||||
- **URL:** Not documented (internal deployment)
|
||||
- **Port:** N/A (static file serving via Apache)
|
||||
- **Service:** N/A
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/hm-o2e-tool
|
||||
|
||||
**Deploy script note:** Sets owner to `vadym.samoilenko` as of commit f1c17c3; ensure this user exists on deployment target.
|
||||
|
||||
## Environment Variables
|
||||
Configuration is loaded from `config.php` at runtime via AJAX (not environment variables):
|
||||
|
||||
- **API credentials** — One2Edit API URL and authentication tokens (stored in config.php, not version-controlled)
|
||||
- **Hardcoded asset IDs:**
|
||||
- `68626a50da85f5bf560161ed` — H&M sRGB Logo
|
||||
- `68626a4a0eb4d535b80789cb` — H&M CMYK Logo
|
||||
- **Campaign folder structure** — Maps campaign names to folder IDs in One2Edit
|
||||
- **Performance delays:**
|
||||
- Relink delay: 5000ms
|
||||
- Document delay: 2000ms
|
||||
- Status check delay: 500ms
|
||||
|
||||
**Note:** Credentials are environment-agnostic; config.php serves as single source of truth.
|
||||
|
||||
## API / Endpoints
|
||||
All calls go to **One2Edit REST API**. Key operations:
|
||||
|
||||
| Method | Endpoint | Purpose |
|
||||
|--------|----------|---------|
|
||||
| POST | `document.session.open` | Create edit session for document |
|
||||
| POST | `document.session.save` | Commit changes to document |
|
||||
| POST | `document.session.close` | Release session lock |
|
||||
| POST | `document.relink_image` | Replace image in document |
|
||||
| POST | `document.export` | Export document (INDD/PDF) |
|
||||
| GET | `asset.search` | Find images in campaign folder by name |
|
||||
| GET | `document.images` | Retrieve image list and link status |
|
||||
|
||||
**Session ID flow:**
|
||||
1. `document.session.open(docId)` → returns `sessionId`
|
||||
2. Use `sessionId` in all subsequent operations
|
||||
3. `document.session.save(sessionId)` → commits
|
||||
4. `document.session.close(sessionId)` → unlocks
|
||||
|
||||
## Known Issues
|
||||
- **Session auto-expiry:** Sessions expire after inactivity; check debug log for session IDs if operations fail
|
||||
- **CMYK detection:** Only files ending with `39L_TAC330.tif` detected as CMYK; edge cases may default to RGB
|
||||
- **Fuzzy matching limitations:** Generic image search may fail on highly non-standard naming conventions
|
||||
- **No offline support:** Requires live One2Edit API connection; no caching
|
||||
- **Debug log size:** Not cleared; may grow large on extended sessions (use browser devtools to clear)
|
||||
|
||||
## Git
|
||||
- **Remote:** `git@bitbucket.org:zlalani/hm-o2e-tool.git`
|
||||
- **Branch strategy:** Not documented
|
||||
- **CI/CD:** None documented; manual deployment via `deploy.sh`
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
|
|
@ -35,4 +150,4 @@ One2Edit API tool for H&M documents. Automated image relinking (by File or by Fo
|
|||
|
||||
## Related
|
||||
- [[hm_ems_report/HM EMS Report Tool]]
|
||||
- [[3m-portal/3M OMG Portal]] (same One2Edit platform)
|
||||
- [[3m-portal/3M OMG Portal]] (same One2Edit platform)
|
||||
|
|
@ -2,27 +2,140 @@
|
|||
name: "H&M EMS Product Review Tool"
|
||||
client: H&M
|
||||
status: active
|
||||
server: optical-web-1
|
||||
tech: [Python, HTML/JS, JSON]
|
||||
server: ai-sandbox.oliver.solutions
|
||||
tech: [Python, Flask, HTML5, JavaScript, JSON, Gunicorn, Apache]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/hm_ems_report
|
||||
deploy: python server.py
|
||||
url:
|
||||
deploy: sudo ./deploy.sh
|
||||
url: https://ai-sandbox.oliver.solutions/hm-ems-report
|
||||
tags: [hm, ems, product-review, multilanguage, json]
|
||||
created: 2026-04-14
|
||||
port: 5000
|
||||
service: hm-ems-report
|
||||
---
|
||||
|
||||
## Overview
|
||||
Web review tool for H&M EMS product data. Clients review product names and prices across up to 4 languages simultaneously, edit inline, approve → saves back to master JSON. Full change log. Campaign images with enlarge preview.
|
||||
H&M EMS Product Review Tool is a web-based application for reviewing and editing H&M product data (names, prices) across multiple languages. Clients access a single-page UI to compare product information side-by-side in up to 4 target languages, make inline edits, approve changes, and automatically save them back to master JSON files with full change logging. It operates in two modes: as a live Flask server with API endpoints or as a static HTML generator for standalone use.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** HTML/JS
|
||||
- **Backend:** Python
|
||||
- **Data:** Master JSON (`Master_Json/` folder)
|
||||
- **Infrastructure:** No Docker
|
||||
- **Frontend:** HTML5, vanilla JavaScript (single-page app, no framework)
|
||||
- **Backend:** Python 3, Flask 2.0+
|
||||
- **Database:** JSON files (flat file storage with changelog tracking)
|
||||
- **Infrastructure:** Apache reverse proxy + Gunicorn (production); direct Flask (development)
|
||||
- **AI/ML:** N/A
|
||||
- **Key libraries:** Flask, Gunicorn, file-based locking for thread-safe JSON operations
|
||||
|
||||
## Architecture
|
||||
|
||||
### Two Operation Modes
|
||||
1. **Server Mode** (`server.py`): Live Flask web app serving API endpoints and static files with real-time editing and approval
|
||||
2. **Static Generator Mode** (`html_generator.py`): Generates self-contained HTML files with embedded data for standalone use
|
||||
|
||||
### Core Components
|
||||
- **server.py**: Flask backend
|
||||
- Thread-safe JSON read/write using file locks
|
||||
- Change logging to `*_changelog.json` files
|
||||
- Image serving with path traversal protection
|
||||
- API endpoints for campaigns, approvals, and saves
|
||||
|
||||
- **html_generator.py**: Shared utilities (60+ language/locale mappings, image path helpers)
|
||||
|
||||
- **static/index.html**: Client-side SPA with:
|
||||
- Dynamic column management (up to 4 target languages)
|
||||
- Inline editing with real-time change tracking
|
||||
- Approve/unapprove workflow with field locking
|
||||
- Multi-image support with click-to-enlarge
|
||||
|
||||
### Data Flow
|
||||
```
|
||||
Master_Json/{campaign}.json
|
||||
↓ (loaded by Flask or generator)
|
||||
Campaign data grouped by Article ID
|
||||
↓ (processed with image mapping)
|
||||
Frontend table with editable fields
|
||||
↓ (on approve)
|
||||
POST /api/approve → updates master JSON + appends changelog
|
||||
```
|
||||
|
||||
### Image Handling
|
||||
- **Source:** `.tif` files listed in JSON `Filename` field (comma-separated for multi-image articles)
|
||||
- **Conversion:** `.tif` → `.jpg` (via helper functions)
|
||||
- **Location:** `campaign_images/{year}/{campaign}/Automation_LR/`
|
||||
- **Year derivation:** First 4 digits of `Season` field (e.g., "202502" → "2025")
|
||||
- **Campaign prefix:** Text before first underscore in JSON filename (e.g., "1022A.json" → "1022A")
|
||||
|
||||
### Approval & Changelog System
|
||||
- **Approval state:** Tracked per Article ID + Language combination
|
||||
- **Changelog format:** `{campaign}_changelog.json` stores timestamped entries with Article ID, Language, Country, Product ID, action ("approve"/"unapprove"), and edits array
|
||||
- **Approval persistence:** On load, changelog is replayed to rebuild approval state
|
||||
- **Thread safety:** All JSON writes use `file_lock` to prevent concurrent modification
|
||||
|
||||
## Dev Commands
|
||||
```bash
|
||||
# Setup virtual environment
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Run locally on http://localhost:5000
|
||||
python3 server.py
|
||||
|
||||
# Run with Gunicorn (production-like)
|
||||
gunicorn server:app --bind 0.0.0.0:5000 --workers 4
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `python server.py`
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/hm_ems_report`
|
||||
- **Server:** ai-sandbox.oliver.solutions (Ubuntu with Apache)
|
||||
- **Deploy:** `sudo ./deploy.sh` from `/opt/hm_ems_report`
|
||||
- **URL:** https://ai-sandbox.oliver.solutions/hm-ems-report
|
||||
- **Port:** 5000 (internal Gunicorn); Apache reverse proxies at :443
|
||||
- **Service:** `hm-ems-report` (systemd service)
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/hm_ems_report
|
||||
|
||||
### Architecture
|
||||
- **User Request (HTTPS)** → Apache (:443) on ai-sandbox.oliver.solutions
|
||||
- `/hm-ems-report/` (static files) → `/var/www/html/hm-ems-report/`
|
||||
- `/hm-ems-report/api/*` (proxy) → Gunicorn :5000 → Flask
|
||||
- `/hm-ems-report/images/*` → `/opt/hm-ems-data/campaign_images/`
|
||||
|
||||
### Setup (First Time)
|
||||
1. Clone to `/opt/hm_ems_report`
|
||||
2. Create data directories: `/opt/hm-ems-data/{Master_Json,campaign_images,backups}` and `/var/log/hm-ems`
|
||||
3. Set ownership: `sudo chown -R vadym.samoilenko:vadym.samoilenko /opt/hm-ems-data /var/www/html/hm-ems-report /var/log/hm-ems`
|
||||
4. Upload campaign JSON files to `/opt/hm-ems-data/Master_Json/`
|
||||
5. Upload campaign images to `/opt/hm-ems-data/campaign_images/{year}/{campaign}/Automation_LR/`
|
||||
6. Configure Apache: add ProxyPass directives for `/hm-ems-report/api` → localhost:5000
|
||||
7. Run `sudo ./deploy.sh`
|
||||
|
||||
## Environment Variables
|
||||
- `IMAGE_BASE_PATH` — Root folder containing `{year}/{campaign}/Automation_LR/` images (default: `./campaign_images`)
|
||||
- `MASTER_JSON_DIR` — Folder containing campaign JSON files (default: `./Master_Json`)
|
||||
- `PORT` — Server port for `python3 server.py` (default: `5000`)
|
||||
- `WORKERS` — Number of Gunicorn workers (default: `4`)
|
||||
- `SECRET_KEY` — Flask secret key for sessions (must be set in production)
|
||||
- `DEPLOY_USER` — User for deployment ownership (default: `vadym.samoilenko`)
|
||||
- `WEB_DIR` — Static files deployment path (default: `/var/www/html/hm-ems-report`)
|
||||
- `APP_DIR` — Application code path (default: `/opt/hm_ems_report`)
|
||||
- `DATA_DIR` — Data directory path (default: `/opt/hm-ems-data`)
|
||||
|
||||
## API Endpoints
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| GET | `/` | Review UI (single-page app) |
|
||||
| GET | `/api/files` | List available campaign JSON files |
|
||||
| GET | `/api/load/<filename>` | Load campaign data, images, languages, and approval state |
|
||||
| POST | `/api/approve` | Approve/unapprove a record, save edits to master JSON, log changes |
|
||||
|
||||
## Known Issues
|
||||
- Application expects consistent JSON structure with `Season`, `Filename`, and article ID fields
|
||||
- Image conversion from `.tif` to `.jpg` is handled by path mapping; actual `.tif` files must exist in `Automation_LR/`
|
||||
- No built-in authentication (recent commits added simple login but status unclear); verify `.env` `SECRET_KEY` is configured in production
|
||||
- Change log replay on startup could be slow with very large changelog files (no pagination/archiving documented)
|
||||
|
||||
## Git
|
||||
- **Remote:** git@bitbucket.org:zlalani/hm_ems_report.git
|
||||
- **Recent commits:** Focus on deployment/Apache configuration (ProxyFix, redirects, login auth), image path fixes, and systemd service setup
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
|
|
@ -35,4 +148,4 @@ Web review tool for H&M EMS product data. Clients review product names and price
|
|||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
|
||||
## Related
|
||||
- [[hm-o2e-tool/HM O2E Tool]]
|
||||
- [[hm-o2e-tool/HM O2E Tool]]
|
||||
|
|
@ -3,25 +3,148 @@ name: "Homepage Dashboard"
|
|||
client: Oliver Internal
|
||||
status: active
|
||||
server: optical-dev
|
||||
tech: [Node.js, Docker, YAML]
|
||||
tech: [Next.js, React, JavaScript, Tailwind CSS, Vitest, Docker, Kubernetes]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/homepage
|
||||
deploy: docker compose up --build
|
||||
url:
|
||||
deploy: docker-compose up
|
||||
url: https://optical-dev.oliver.solutions/homepage
|
||||
tags: [oliver, dashboard, homepage, selfhosted]
|
||||
created: 2026-04-14
|
||||
port: 3001
|
||||
---
|
||||
|
||||
## Overview
|
||||
Self-hosted application dashboard with 100+ service integrations, multi-language support. Configured via YAML files or Docker label discovery. Fully static, fast, proxied.
|
||||
|
||||
Homepage is a self-hosted application dashboard built with Next.js that aggregates 100+ service integrations into a single customizable interface. It solves the problem of managing multiple self-hosted applications by providing a unified, fast, secure entry point with Docker/Kubernetes autodiscovery, YAML-based configuration, and full API request proxying to protect credentials. Designed for privacy-conscious users running containerized infrastructure.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** Node.js (static)
|
||||
- **Config:** YAML or Docker labels
|
||||
- **Infrastructure:** Docker + docker-compose
|
||||
|
||||
- **Frontend:** Next.js 16, React 19, Tailwind CSS, Headless UI components
|
||||
- **Backend:** Next.js API routes, Node.js runtime
|
||||
- **Database:** None (fully static with YAML configuration)
|
||||
- **Infrastructure:** Docker, Docker Compose, Kubernetes API support, dockerode for container integration
|
||||
- **Internationalization:** next-i18next with 40+ language translations
|
||||
- **Key libraries:** SWR (data fetching), Recharts (charts), React Icons, Winston (logging), js-yaml (config parsing), Luxon (date/time)
|
||||
|
||||
## Architecture
|
||||
|
||||
Homepage follows a **static-first, API-proxy architecture**:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ Browser / User Interface │
|
||||
│ (Static React components + SWR data fetching) │
|
||||
└──────────────────┬──────────────────────────────────┘
|
||||
│
|
||||
┌─────────▼──────────┐
|
||||
│ Next.js Routes │
|
||||
│ /api/widgets/... │
|
||||
│ /api/docker/... │
|
||||
│ /api/config/... │
|
||||
│ /api/kubernetes/ │
|
||||
└─────────┬──────────┘
|
||||
│
|
||||
┌─────────────┼─────────────┐
|
||||
▼ ▼ ▼
|
||||
External Docker Kubernetes
|
||||
Services Socket API
|
||||
(Radarr, (Container (Pod/Service
|
||||
Sonarr, Discovery) Discovery)
|
||||
etc.)
|
||||
```
|
||||
|
||||
**Widget Plugin System** (`src/widgets/`): 162+ service integrations, each a self-contained directory with:
|
||||
- `component.jsx` — React widget UI component
|
||||
- `index.js` — metadata, configuration schema, validation
|
||||
- `logo.svg` — service branding
|
||||
|
||||
**Configuration Layer** (`src/utils/config/`): Parses YAML files from `/config` volume mount. Supports:
|
||||
- Manual service configuration (YAML)
|
||||
- Automatic Docker container discovery via labels
|
||||
- Kubernetes service discovery
|
||||
- Environment variable overrides
|
||||
|
||||
**API Proxying** (`src/pages/api/widgets/[...service].js`): Routes all external service requests server-side, ensuring API credentials never reach the browser. Caches responses via `memory-cache`.
|
||||
|
||||
**Path Aliases** recognized by both Vitest and ESLint:
|
||||
- `components`, `pages`, `styles`, `test-utils`, `utils`, `widgets` → map to `./src/<name>`
|
||||
|
||||
**i18n**: Translations in `public/locales/<lang>/`. Custom formatters for bytes, rates, durations defined in `next-i18next.config.js`.
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
# Install dependencies (pnpm enforced)
|
||||
pnpm install
|
||||
|
||||
# Start dev server (http://localhost:3000)
|
||||
pnpm dev
|
||||
|
||||
# Production build
|
||||
pnpm build
|
||||
|
||||
# Linting
|
||||
pnpm lint
|
||||
|
||||
# Testing
|
||||
pnpm test # Run all tests once (526 test files)
|
||||
pnpm test src/components/foo.test.jsx # Single file
|
||||
pnpm test:watch # Watch mode
|
||||
pnpm test:coverage # Coverage report
|
||||
```
|
||||
|
||||
**Testing conventions:**
|
||||
- Component tests: add `// @vitest-environment jsdom` at top
|
||||
- Utility/API tests: run in Node environment (default)
|
||||
- Use `renderWithProviders()` from `test-utils` for context-dependent components
|
||||
- `vitest.setup.js` globally mocks `next-i18next`
|
||||
|
||||
## Deployment
|
||||
- **Run:** `docker compose up --build`
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/homepage`
|
||||
|
||||
- **Server:** optical-dev (Oliver infrastructure)
|
||||
- **Deploy:** `docker-compose up` (see `docker-compose.yml`)
|
||||
- **URL:** https://optical-dev.oliver.solutions/homepage
|
||||
- **Port:** 3001 (exposed via docker-compose)
|
||||
- **Service:** Docker containerized (no systemd service)
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/homepage
|
||||
|
||||
**Docker setup:**
|
||||
- Mounts `/config` volume for YAML configuration
|
||||
- Mounts `/var/run/docker.sock` for container discovery
|
||||
- Healthcheck via wget to `/api/healthcheck` every 10s
|
||||
- Auto-restart on failure
|
||||
|
||||
## Environment Variables
|
||||
|
||||
- `NEXT_PUBLIC_BASE_PATH` — URL path prefix (e.g., `/homepage`); build arg in Dockerfile
|
||||
- `HOSTNAME` — Server bind address (currently `::` for IPv6)
|
||||
- `HOMEPAGE_ALLOWED_HOSTS` — Host validation for Next.js 15+ (set to `optical-dev.oliver.solutions`)
|
||||
- `NEXT_PUBLIC_*` — Any vars prefixed with this are exposed to the browser
|
||||
|
||||
## API / Endpoints
|
||||
|
||||
- `/api/widgets/[...service]` — Proxied service data endpoints (e.g., `/api/widgets/radarr/movies`)
|
||||
- `/api/docker/*` — Docker container integration, stats, autodiscovery
|
||||
- `/api/kubernetes/*` — Kubernetes Pod/Service discovery
|
||||
- `/api/config` — Serves parsed YAML configuration
|
||||
- `/api/healthcheck` — Health probe endpoint
|
||||
- `/_next/image` — Next.js image optimization
|
||||
|
||||
## Known Issues
|
||||
|
||||
- **No AI-generated PRs** without explicit disclosure (per CONTRIBUTING.md)
|
||||
- **Pre-commit hooks** run Prettier and YAML validation — do not skip with `--no-verify`
|
||||
- **ESLint rules enforce:**
|
||||
- No import cycles (`maxDepth: 1`)
|
||||
- Import ordering via plugin
|
||||
- No `else` after `return`
|
||||
- Bugs tracked in **GitHub Discussions**, not Issues
|
||||
- basePath support recently added (commits 7d80b21c–be5e2a5e); ensure all client-side API calls prepend basePath
|
||||
|
||||
## Git
|
||||
|
||||
- **Remote:** git@bitbucket.org:zlalani/homepage.git
|
||||
- **Latest commits:** Focus on basePath support, deploy widget integration, Docker service discovery, and crowdin translations
|
||||
- **Version:** 1.10.1
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
|
|
@ -31,4 +154,4 @@ Self-hosted application dashboard with 100+ service integrations, multi-language
|
|||
## Change Log
|
||||
| Date | Requested | Changed | Files |
|
||||
|------|-----------|---------|-------|
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
|
|
@ -2,8 +2,8 @@
|
|||
name: "L'Oréal OMG Assistant Global"
|
||||
client: L'Oréal
|
||||
status: active
|
||||
server: baic (web-03)
|
||||
tech: [PHP, Box API, Make.com, Webhook]
|
||||
server: local
|
||||
tech: [PHP, JavaScript, Box API, Azure AD SSO, Mailgun, Make.com Webhooks]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/loreal-global-kickoff
|
||||
deploy: PHP server
|
||||
url:
|
||||
|
|
@ -12,20 +12,136 @@ created: 2026-04-14
|
|||
---
|
||||
|
||||
## Overview
|
||||
PHP web app for L'Oréal campaign asset management:
|
||||
1. **Master Global Asset Submission** — submit Box assets with metadata, auto-populate campaign number from folder hierarchy
|
||||
2. **Global to Local** — transform global campaign CSVs into regional market CSVs
|
||||
|
||||
Webhook integration to Make.com on submission.
|
||||
L'Oréal OMG Assistant Global is a PHP web application for managing L'Oréal campaign assets and regional distribution. It provides two primary tools: Master Global Asset Submission (validating and submitting Box folder assets with metadata) and Global to Local CSV Transformation (converting global campaign data into 16 regional market-specific CSVs). The application serves L'Oréal's internal teams to streamline asset management workflows with comprehensive audit logging, SSO authentication, and email notifications.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend/Backend:** PHP (server-rendered)
|
||||
- **Integrations:** Box API, Make.com webhook
|
||||
- **Infrastructure:** PHP server (MAMP/XAMPP or standalone)
|
||||
- **Frontend:** HTML5, CSS3, JavaScript (vanilla), Montserrat font
|
||||
- **Backend:** PHP 7.4+, Composer
|
||||
- **APIs & Integrations:** Box API (JWT), Azure AD SSO, Make.com webhooks, Mailgun SMTP, OMG API
|
||||
- **Database:** JSON file-based activity logs (no SQL DB required)
|
||||
- **Infrastructure:** Local development (MAMP) or web server (Apache/Nginx)
|
||||
- **Key libraries:** Box PHP SDK, MSAL (Microsoft Authentication), curl for API requests, ZIP file generation
|
||||
|
||||
## Architecture
|
||||
The application follows a modular, request-driven architecture with clear separation of concerns:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Web Interface (HTML/CSS/JS) │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ Master Global Global to Local Activity │
|
||||
│ Asset Submission CSV Transform Logs/Viewer │
|
||||
└──────────────┬──────────────────┬──────────────────┬─────────┘
|
||||
│ │ │
|
||||
┌──────▼──────────────────▼──────────────────▼──────┐
|
||||
│ PHP Application Layer (index.php) │
|
||||
│ • Route handling & session management │
|
||||
│ • SSO/RBAC authentication & user tracking │
|
||||
│ • Activity logging to JSON │
|
||||
└──────┬──────────────────┬──────────────────┬───────┘
|
||||
│ │ │
|
||||
┌────────▼────────┐ ┌──────▼──────┐ ┌──────▼──────┐
|
||||
│ Box API │ │ Make.com │ │ Mailgun │
|
||||
│ (folders, │ │ Webhooks │ │ (email) │
|
||||
│ validation) │ │ (submit) │ │ │
|
||||
└─────────────────┘ └─────────────┘ └─────────────┘
|
||||
│
|
||||
┌────────▼────────────────────┐
|
||||
│ Azure AD SSO / OMG API │
|
||||
│ (auth, business unit lookup)│
|
||||
└─────────────────────────────┘
|
||||
|
||||
Data Storage:
|
||||
• logs/activity_log.json — Audit trail of all user actions
|
||||
• config.php — Application configuration
|
||||
• roles.json — User RBAC assignments (admin/user)
|
||||
```
|
||||
|
||||
**Key Design Decisions:**
|
||||
- **No SQL Database**: Uses JSON file logging for simplicity and portability
|
||||
- **Two-Mode Auth**: SSO disabled in local dev (hardcoded user), enabled in production (Azure AD)
|
||||
- **Stateless Processing**: CSV transformation tracked via multi-stage progress with visual feedback
|
||||
- **Webhook-Based Submission**: Master asset submission delegates to Make.com for external processing
|
||||
- **Comprehensive Audit Trail**: Every action (login, submission, error) logged with timestamp, IP, user agent
|
||||
- **16-Market Localization**: Hard-coded ISO codes ensure consistent regional distribution
|
||||
|
||||
## Dev Commands
|
||||
```bash
|
||||
# Install dependencies
|
||||
composer install
|
||||
|
||||
# Local development server (MAMP or built-in)
|
||||
# Access via browser: http://localhost:8888 (MAMP) or http://localhost:8000 (built-in PHP)
|
||||
php -S localhost:8000
|
||||
|
||||
# View git history
|
||||
git log --oneline -20
|
||||
|
||||
# Run tests (if available)
|
||||
# No test command documented; manual testing via web interface
|
||||
|
||||
# Check server setup
|
||||
php server-setup.php
|
||||
|
||||
# View environment diagnostics
|
||||
php server-diagnostics.php
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** PHP web server
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/loreal-global-kickoff`
|
||||
- **Server:** Local development (MAMP) or custom web server
|
||||
- **Deploy:** Manual file sync or git pull on target server
|
||||
- **URL:** Not yet deployed; development only
|
||||
- **Port:** 8000 (PHP built-in) or 8888 (MAMP)
|
||||
- **Service:** None (stateless web application)
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/loreal-global-kickoff
|
||||
|
||||
## Environment Variables
|
||||
All configuration is managed in `config.php`. Key variables:
|
||||
|
||||
- `sso.enabled` — Toggle Azure AD SSO (false for local dev, true for production)
|
||||
- `sso.local_user.email` — Hardcoded email when SSO disabled (local dev mode)
|
||||
- `box.client_id` / `box.client_secret` / `box.enterprise_id` — Box API JWT credentials
|
||||
- `azure_ad.client_id` / `azure_ad.client_secret` / `azure_ad.tenant_id` — Azure AD SSO credentials
|
||||
- `make_webhook_url` — Make.com webhook endpoint for asset submissions
|
||||
- `mailgun.domain` / `mailgun.api_key` — Mailgun SMTP credentials for email notifications
|
||||
- `omg_api.enabled` — Toggle OMG API lookup for business unit (optional)
|
||||
- `omg_api.api_key` — OMG API authentication key (X-API-Key header)
|
||||
|
||||
## Activity Logging & Audit Trail
|
||||
All user actions are logged to `logs/activity_log.json` with:
|
||||
- Timestamp (ISO 8601)
|
||||
- User email
|
||||
- Action type (login, asset_submission, csv_transform, error, etc.)
|
||||
- Status (success, error)
|
||||
- IP address & user agent
|
||||
- Action-specific data (folder ID, file count, error details, etc.)
|
||||
|
||||
The Activity Logs viewer (`/activity-logs`) displays:
|
||||
- Filterable log entries by action type and user
|
||||
- Statistics dashboard (total actions, success rates, error count, unique users)
|
||||
- Export to CSV functionality
|
||||
|
||||
## Endpoints
|
||||
- **GET `/`** — Main application (route-driven via `$_GET['page']`)
|
||||
- **GET `/?page=master-submission`** — Master Global Asset Submission form
|
||||
- **GET `/?page=csv-transform`** — Global to Local CSV transformation tool
|
||||
- **GET `/?page=activity-logs`** — Activity logs viewer & dashboard
|
||||
- **POST `/api/get-box-folder`** — Fetch Box folder structure (AJAX)
|
||||
- **POST `/api/process-csv`** — Process CSV transformation (AJAX, multi-stage)
|
||||
- **POST `/api/submit-assets`** — Submit asset data to Make.com webhook
|
||||
- **POST `/api/login`** — Azure AD SSO login (if enabled)
|
||||
|
||||
## Known Issues
|
||||
- **OMG API Optional**: OMG business unit lookup can fail in testing; fallback to manual entry
|
||||
- **Email Delivery**: Mailgun SMTP may require firewall/DNS configuration in production
|
||||
- **SSO Configuration**: Azure AD redirect_uri must exactly match registered URI (including trailing slash)
|
||||
- **Box API Rate Limits**: Real-time folder validation may be slow on large hierarchies
|
||||
- **No Unit Tests**: Application tested manually via web interface only
|
||||
|
||||
## Git
|
||||
- **Remote:** git@bitbucket.org:zlalani/loreal-global-kickoff.git
|
||||
- **Branch Strategy:** Single main branch (development in progress)
|
||||
- **Recent Activity:** SSO/RBAC implementation (commit 53e9365), user login logging, activity dashboard enhancements
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
|
|
@ -39,4 +155,4 @@ Webhook integration to Make.com on submission.
|
|||
|
||||
## Related
|
||||
- [[loreal-sla-calculator/Loreal SLA Calculator]]
|
||||
- [[Loreal/Loreal automation]]
|
||||
- [[Loreal/Loreal automation]]
|
||||
|
|
@ -3,29 +3,199 @@ name: "L'Oréal SLA Calculator"
|
|||
client: L'Oréal
|
||||
status: active
|
||||
server: optical-web-1
|
||||
tech: [HTML/JS, Docker, Node.js]
|
||||
tech: [JavaScript, Node.js, Express, PostgreSQL, HTML/CSS, Docker]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/loreal-sla-calculator
|
||||
deploy: docker compose up --build
|
||||
deploy: docker-compose up
|
||||
url:
|
||||
tags: [loreal, sla, timeline, gantt, ecom]
|
||||
created: 2026-04-14
|
||||
port: 3100
|
||||
db: PostgreSQL
|
||||
---
|
||||
|
||||
## Overview
|
||||
SLA calculator for eCom Content Factory PMs. Replaces Excel. 4-step wizard: select brief type → configure stages → enter dates → view Gantt + verdict.
|
||||
|
||||
Two versions:
|
||||
- `index.html` — Full Calculator (9 brief types × 8 stages, Gantt, CSV/iCal export)
|
||||
- `client.html` — Client Estimator (4 inputs, SLA defaults)
|
||||
The **loreal-sla-calculator** is a web-based SLA (Service Level Agreement) calculator for L'Oréal's eCom Content Factory that replaces legacy Excel-based workflows. It provides two interfaces: a **Full PM Calculator** (`index.html`) with a 4-step wizard for project managers to estimate timelines and determine deadline feasibility, and a **Client Estimator** (`client.html`) with a simplified single-page form for clients using SLA defaults. The calculator computes project timelines across 8 configurable stages (Production, Translation, Syndication, Opera Upload, etc.), accounts for complexity, asset volume, revision rounds, and market approval days, and outputs results as timeline breakdowns, Gantt charts, and exportable calendars (CSV/iCal).
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** HTML/JS + Gantt rendering
|
||||
- **Infrastructure:** Docker + docker-compose
|
||||
|
||||
- **Frontend:** JavaScript (vanilla), HTML5, CSS3, responsive design
|
||||
- **Backend:** Node.js, Express.js
|
||||
- **Database:** PostgreSQL 16 (Alpine Linux)
|
||||
- **Infrastructure:** Docker Compose, systemd-compatible deployment
|
||||
- **Configuration:** JSON-based business rules (non-developer editable)
|
||||
- **Key libraries:** Nodemon (dev), MSAL (Azure SSO), Analytics tracking
|
||||
|
||||
## Architecture
|
||||
|
||||
The project uses a **client-server hybrid model**:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Frontend (Client-Side) │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ index.html (Full PM Calculator) client.html (Estimator)│
|
||||
│ • 4-step wizard • Single-page form │
|
||||
│ • Gantt chart visualization • Simplified inputs │
|
||||
│ • CSV/iCal export • Default values │
|
||||
│ • Dark mode toggle • Market Brief Advisor │
|
||||
│ • Usage analytics tracking • SSO integrated │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Shared Configuration Layer │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ config.json (Business Rules) script.js / client-script.js
|
||||
│ • 9 brief types & stage matrix • Calculation engines │
|
||||
│ • Production cross-ref table • Form dynamics │
|
||||
│ • Syndication cross-ref table • Timeline generation │
|
||||
│ • Translation word ranges • Export logic │
|
||||
│ • Default values (revisions, etc.) • Dark mode persistence│
|
||||
│ • Tooltips & descriptions • Analytics events │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Backend (Node.js/Express) │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ server/index.js (Express app) │
|
||||
│ • Serves static HTML/CSS/JS │
|
||||
│ • Azure SSO authentication (MSAL) │
|
||||
│ • Analytics event logging API │
|
||||
│ • User session management │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Database (PostgreSQL 16) │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ • Analytics events table │
|
||||
│ • User activity logs │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Key Design Decisions:**
|
||||
- **Config-driven:** All business rules centralized in `config.json` so PMs/non-devs can edit without code changes
|
||||
- **Dual interfaces:** Full calculator for power users (PMs), simplified estimator for clients
|
||||
- **Business-day math:** All calculations use Monday-Friday only (excludes weekends)
|
||||
- **Syndication buffer:** When syndication is active, 10 business days auto-added to suggested go-live date
|
||||
- **Parallel stages:** Stages 4 & 5 (Translation PDP + Asset) share one input set and run concurrently
|
||||
- **Locked Opera Upload:** Always enabled (`alwaysActive: true`); users cannot deactivate it
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
npm install
|
||||
|
||||
# Start dev server with hot reload (runs on port 3100)
|
||||
npm run dev
|
||||
|
||||
# Start production server
|
||||
npm start
|
||||
|
||||
# Run database migrations
|
||||
npm run migrate
|
||||
|
||||
# Build Docker image and run locally
|
||||
docker-compose up
|
||||
|
||||
# View logs from Docker container
|
||||
docker-compose logs -f app
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `docker compose up --build`
|
||||
|
||||
- **Server:** Docker Compose (self-hosted on `/opt/loreal-sla-calculator/`)
|
||||
- **Deploy:** `docker-compose up -d` (restarts services unless stopped)
|
||||
- **URL:** Accessible via configured domain/reverse proxy (port 3100 internally)
|
||||
- **Port:** 3100
|
||||
- **Service:** Managed via Docker Compose (postgres and app services)
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/loreal-sla-calculator`
|
||||
|
||||
**Deploy Notes:**
|
||||
- Environment variables loaded from `/opt/loreal-sla-calculator/server/.env`
|
||||
- PostgreSQL data persists via `postgres_data` Docker volume
|
||||
- Health checks ensure postgres is ready before app starts
|
||||
- Requires `.env` file with `POSTGRES_USER`, `POSTGRES_DB`, and other secrets
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Key environment variables (loaded from `.env`):
|
||||
|
||||
- `PORT` — Server port (default: 3100)
|
||||
- `NODE_ENV` — Set to `production` in production; `development` for local dev
|
||||
- `SERVE_STATIC` — Whether to serve static files (set to `false` if behind reverse proxy)
|
||||
- `POSTGRES_USER` — PostgreSQL username
|
||||
- `POSTGRES_DB` — PostgreSQL database name
|
||||
- `POSTGRES_PASSWORD` — PostgreSQL password
|
||||
- `DATABASE_URL` — Full PostgreSQL connection string (if used by app)
|
||||
- `MSAL_*` — Azure SSO configuration (MSAL client ID, tenant, etc.)
|
||||
- `ANALYTICS_DB_*` — Analytics table connection details (if separate)
|
||||
|
||||
## Calculation Logic
|
||||
|
||||
**Core Timeline Calculation:**
|
||||
|
||||
For each **active stage**, the system computes:
|
||||
```
|
||||
Stage Duration = Handover (1 day)
|
||||
+ WIP Days (from cross-ref table)
|
||||
+ Market Approval Days
|
||||
+ (Revision Days × Revision Rounds)
|
||||
```
|
||||
|
||||
**Cross-Reference Tables (in config.json):**
|
||||
- **Production:** Days indexed by [complexity level] × [asset volume] (up to 400 assets)
|
||||
- **Syndication:** Days indexed by [complexity level] × [EAN volume]
|
||||
- **Translation:** Days indexed by [word count bucket] (e.g., <2K, 2K-5K, 5K+)
|
||||
- **Opera Upload:** Days indexed by [asset volume]
|
||||
|
||||
**Special Rules:**
|
||||
- **Syndication buffer:** When syndication stage is active, +10 business days added to suggested go-live
|
||||
- **Stages 4 & 5 parallel:** Both stages use the same complexity/asset inputs and run concurrently (max of both durations)
|
||||
- **Opera Upload always on:** Cannot be toggled off by users; mandatory for all brief types
|
||||
- **Business days only:** Uses Monday-Friday calculations (skips weekends)
|
||||
|
||||
## Known Issues
|
||||
|
||||
- Static work type sub-selector and brief type override dropdown added (commit a1abbbc) — verify UI/UX in edge cases
|
||||
- Decimal feedback days supported (0.1 step) but may show precision rounding on export
|
||||
- Analytics dashboard accessible only via direct URL (removed from nav links per commit 28fc722)
|
||||
- MSAL CDN migration to jsdelivr (commit 8843af6) — ensure cache-busting if issues arise
|
||||
- SSO redirect URI must match Azure app registration exactly (no trailing slash per commit 53a73cb)
|
||||
|
||||
## Git
|
||||
|
||||
- **Remote:** `git@bitbucket.org:zlalani/loreal-sla-calculator.git`
|
||||
- **Recent work:** SSO fixes, analytics dashboard, static imagery & work type selectors, syndication defaults, market brief advisor
|
||||
- **Latest commit:** 53a73cb — Fix SSO redirect URI (remove trailing slash)
|
||||
|
||||
## References & Notes
|
||||
|
||||
**For Editing Business Rules:**
|
||||
Simply edit `config.json` to adjust:
|
||||
- Brief type definitions & stage application matrix
|
||||
- Production/Syndication/Translation cross-reference tables
|
||||
- Complexity levels, asset volume ranges, word count buckets
|
||||
- Default revision rounds, market approval days, revision days per round
|
||||
- Help text, tooltips, and syndication buffer value
|
||||
|
||||
**For Adding New Features:**
|
||||
- PM calculator logic: `script.js` (4-step wizard, Gantt, export)
|
||||
- Client estimator logic: `client-script.js` (simplified form, defaults)
|
||||
- Backend routes: `server/index.js` (SSO, analytics, static serving)
|
||||
- DB schema: `server/db/migrate.js` (analytics tables)
|
||||
|
||||
**Testing Checklist:**
|
||||
- [ ] All 9 brief types load and display correct stages
|
||||
- [ ] Complexity/asset volume lookups match cross-ref tables
|
||||
- [ ] Syndication +10 day buffer applied correctly
|
||||
- [ ] Gantt chart renders across full timeline
|
||||
- [ ] CSV/iCal exports contain all key milestone dates
|
||||
- [ ] Dark mode persists to localStorage
|
||||
- [ ] Mobile layout responsive on <768px screens
|
||||
- [ ] SSO redirects to correct Azure tenant
|
||||
- [ ] Analytics events logged to database
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
**Done:** Added to Obsidian second brain.
|
||||
|
|
@ -37,4 +207,4 @@ Two versions:
|
|||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
|
||||
## Related
|
||||
- [[loreal-global-kickoff/Loreal Global Kickoff]]
|
||||
- [[loreal-global-kickoff/Loreal Global Kickoff]]
|
||||
|
|
@ -1,29 +1,115 @@
|
|||
---
|
||||
name: "LUSA Back Planner"
|
||||
client: LUSA
|
||||
client: L'Oréal USA
|
||||
status: active
|
||||
server: optical-web-1
|
||||
tech: [React, TypeScript, Vite, Tailwind, shadcn/ui, pdfjs, jsPDF]
|
||||
tech: [React, TypeScript, Vite, Tailwind CSS, pdfjs-dist, jsPDF, Azure AD (MSAL)]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/lusa-back-planner
|
||||
deploy: npm run dev
|
||||
url:
|
||||
deploy: Apache virtual host + scp
|
||||
url: https://ai-sandbox.oliver.solutions/lusa-Back-Planner
|
||||
tags: [lusa, timeline, planner, pdf, react, typescript]
|
||||
created: 2026-04-14
|
||||
port: 80
|
||||
service: apache2
|
||||
---
|
||||
|
||||
## Overview
|
||||
Production timeline generator. Set a deadline, pick a production lane → instant timeline. PDF brief parsing, change requests, PDF export.
|
||||
|
||||
LUSA Back Planner is a client-side React SPA that generates production timelines for L'Oréal USA brand teams. Users upload a PDF brief, select a production lane (Rush/Express/Production/Specialist), and receive an instant workback timeline with milestones tied to the deadline. The app supports PDF brief parsing to auto-extract dates and brands, change request workflows, and PDF export. Authentication is enforced via Azure AD SSO (MSAL), making it accessible only to authorized L'Oréal team members.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** React 19 + TypeScript + Vite
|
||||
- **Styling:** Tailwind CSS + shadcn/ui
|
||||
- **PDF:** pdfjs-dist (parsing) + jsPDF + html2canvas (export)
|
||||
- **Infrastructure:** No Docker
|
||||
|
||||
- **Frontend:** React 18 + TypeScript + Vite
|
||||
- **Backend:** None (client-side only)
|
||||
- **Database:** None (no persistence)
|
||||
- **Infrastructure:** Apache 2.4 on Ubuntu Server; deployed as static SPA with FallbackResource fallback
|
||||
- **AI/ML:** None
|
||||
- **Key libraries:** pdfjs-dist (PDF parsing), jsPDF + html2canvas (PDF export), @azure/msal-react (Azure AD SSO), shadcn/ui (Radix UI components), date-fns (date math), react-router-dom
|
||||
|
||||
## Architecture
|
||||
|
||||
The app is a client-side SPA with no backend. Authentication wraps the entire application via `AuthGuard` (MSAL redirect flow); unauthenticated users see a login page with "Sign in with Microsoft" button.
|
||||
|
||||
**Data Flow:**
|
||||
1. User authenticates via Azure AD SSO → MSAL handles redirect/popup flow
|
||||
2. User uploads PDF brief → `parse-brief.ts` extracts due date, brand, and lane suggestion using pdfjs-dist + regex/proximity scoring
|
||||
3. User confirms production lane + deadline → `templates.ts:generateTimeline()` creates milestones by subtracting business days (accounting for federal holidays and weekends via `workdays.ts`)
|
||||
4. `TimelineView` displays milestones, supports inline editing, and exports to PDF via jsPDF + html2canvas
|
||||
5. Change requests: upload existing timeline PDF → `parse-timeline-pdf.ts` extracts milestones → user picks update reason → new PDF exported
|
||||
|
||||
**Component Tree:**
|
||||
```
|
||||
main.tsx
|
||||
└─ AuthGuard (MSAL auth check; wraps entire app)
|
||||
└─ MsalProvider
|
||||
└─ App (BrowserRouter with routes)
|
||||
├─ Index.tsx (main page; form state orchestrator)
|
||||
├─ TimelineView.tsx (timeline display & PDF export)
|
||||
└─ ChangeRequestTab.tsx (change request workflow)
|
||||
```
|
||||
|
||||
**Key Modules:**
|
||||
- `src/lib/templates.ts` — Lane definitions (4 types) + `generateTimeline()` logic
|
||||
- `src/lib/workdays.ts` — Business day math: federal holidays, `subtractWorkdays()`, `countWorkdays()`
|
||||
- `src/lib/parse-brief.ts` — PDF text extraction + keyword proximity + multi-regex date parsing
|
||||
- `src/lib/sla-data.ts` — Asset type → duration mappings (90+ types)
|
||||
- `src/lib/brands.ts` — 38 L'Oréal brand definitions
|
||||
- `src/lib/msal-config.ts` — MSAL instance & login config (env-based)
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
npm install # Install dependencies
|
||||
npm run dev # Dev server at http://localhost:5173
|
||||
npm run build # Production build → dist/
|
||||
npm run build:dev # Development build
|
||||
npm run preview # Preview production build
|
||||
npm run lint # ESLint check
|
||||
npm run test # Vitest (single run)
|
||||
npm run test:watch # Vitest (watch mode)
|
||||
npm run test -- src/test/example.test.ts # Run single test file
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Dev:** `npm run dev`
|
||||
- **Build:** `npm run build`
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/lusa-back-planner`
|
||||
|
||||
- **Server:** optical-web-1 (Ubuntu Server with Apache 2.4)
|
||||
- **Deploy:** Manual via `scp` + Apache config; see README.md for detailed steps
|
||||
- **URL:** https://ai-sandbox.oliver.solutions/lusa-Back-Planner
|
||||
- **Port:** 80 (HTTP) / 443 (HTTPS with Let's Encrypt)
|
||||
- **Service:** Apache 2 (`apache2`)
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/lusa-back-planner
|
||||
|
||||
**Deploy Steps:**
|
||||
1. `npm install && npm run build` (creates `dist/`)
|
||||
2. `scp -r dist/ user@optical-web-1:/var/www/backplanner`
|
||||
3. Create Apache VirtualHost at `/etc/apache2/sites-available/backplanner.conf` (see README.md)
|
||||
4. `sudo a2ensite backplanner.conf && sudo systemctl reload apache2`
|
||||
5. (Optional) Enable HTTPS: `sudo certbot --apache -d ai-sandbox.oliver.solutions`
|
||||
|
||||
**Subdirectory Deploy:** If hosting under a path (e.g., `/backplanner/`), set `base: "/backplanner/"` in `vite.config.ts` before building, then use Apache `Alias` instead of VirtualHost.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
All vars must use `VITE_` prefix to be accessible in the browser.
|
||||
|
||||
- `VITE_AZURE_TENANT_ID` — Azure AD tenant ID (L'Oréal's tenant)
|
||||
- `VITE_AZURE_CLIENT_ID` — Azure AD application client ID; different for dev vs. production
|
||||
- `VITE_AZURE_REDIRECT_URI` — Post-auth redirect URL; must match Azure AD app registration exactly (e.g., `https://ai-sandbox.oliver.solutions/lusa-Back-Planner` for production, `http://localhost:8888/lusa-Back-Planner` for local dev with port 8888)
|
||||
|
||||
Copy `.env.example` to `.env.local` and fill in values. Dev and production use different client IDs but same tenant.
|
||||
|
||||
## Known Issues
|
||||
|
||||
- **MSAL redirect URI mismatch:** The `VITE_AZURE_REDIRECT_URI` must match exactly in Azure AD app registration. Recent commits (4d04d4b, 51bfd5b) fixed redirect URI bugs by adding `basename` to `BrowserRouter` and explicitly setting `redirectUri` in `loginRequest`.
|
||||
- **localStorage key iteration crashes:** Fixed in f1061f5; MSAL now safely iterates localStorage.
|
||||
- **interaction_in_progress errors:** Resolved in 03fd4be by clearing MSAL state on reload; also switched from redirect flow to popup flow (2e3a72c).
|
||||
- No documented limitations on timeline generation, PDF parsing, or asset type coverage.
|
||||
|
||||
## Git
|
||||
|
||||
- **Remote:** `git@bitbucket.org:zlalani/lusa-back-planner.git`
|
||||
- **Last 20 commits:** Focus on MSAL authentication fixes, deploy script improvements, and initial app setup
|
||||
- **Current branch:** Presumed `main` (no branch info in git log)
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
|
|
@ -33,4 +119,4 @@ Production timeline generator. Set a deadline, pick a production lane → instan
|
|||
## Change Log
|
||||
| Date | Requested | Changed | Files |
|
||||
|------|-----------|---------|-------|
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
|
|
@ -2,20 +2,93 @@
|
|||
name: "MD to Word Converter"
|
||||
client: Oliver Internal
|
||||
status: active
|
||||
tech: [Python]
|
||||
tech: [Python, python-docx, markdown, BeautifulSoup4, Pillow]
|
||||
local_path: "/Users/ai_leed/Documents/Projects/Oliver/md to word"
|
||||
deploy: python
|
||||
deploy: pip install
|
||||
url:
|
||||
tags: [oliver, markdown, word, converter, utility]
|
||||
created: 2026-04-14
|
||||
server: local
|
||||
---
|
||||
|
||||
## Overview
|
||||
Utility for converting Markdown files to Word (.docx). Python-based. No README.
|
||||
md-to-word is a Python CLI tool that converts Markdown documents to Word (.docx) format with support for Mermaid chart rendering. It parses Markdown syntax, converts it to structured Word document elements, and embeds rendered Mermaid diagrams as images. This is useful for teams that author content in Markdown but need to distribute or collaborate in Microsoft Word format while preserving rich formatting and diagrams.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** N/A (CLI tool)
|
||||
- **Backend:** Python 3.9+
|
||||
- **Database:** N/A
|
||||
- **Infrastructure:** Local/Command-line
|
||||
- **AI/ML:** N/A
|
||||
- **Key libraries:**
|
||||
- `python-docx` — Word document generation
|
||||
- `markdown` — Markdown parsing
|
||||
- `beautifulsoup4` — HTML parsing (intermediate format)
|
||||
- `Pillow` — Image processing for diagram rendering
|
||||
|
||||
## Architecture
|
||||
The converter works as a single-pipeline tool:
|
||||
|
||||
```
|
||||
Markdown file (.md)
|
||||
↓
|
||||
Markdown parser
|
||||
↓
|
||||
HTML intermediate
|
||||
↓
|
||||
BeautifulSoup parsing
|
||||
↓
|
||||
Mermaid diagram detection & rendering
|
||||
↓
|
||||
python-docx document builder
|
||||
↓
|
||||
Word file (.docx)
|
||||
```
|
||||
|
||||
**Design flow:**
|
||||
1. Read Markdown source file
|
||||
2. Parse Markdown to HTML using the `markdown` library
|
||||
3. Parse HTML with BeautifulSoup to extract structure and content
|
||||
4. Detect Mermaid code blocks and render them to PNG images via Mermaid CLI (or similar)
|
||||
5. Build a Word document programmatically using `python-docx`
|
||||
6. Insert rendered diagrams as images at appropriate locations
|
||||
7. Write final .docx file
|
||||
|
||||
## Dev Commands
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Install in development mode
|
||||
pip install -e .
|
||||
|
||||
# Run conversion (after installation)
|
||||
convert input.md output.docx
|
||||
|
||||
# Run directly from module
|
||||
python -m md_to_word_converter input.md output.docx
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `python convert.py` (or similar entry point)
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/md to word`
|
||||
- **Server:** local
|
||||
- **Deploy:** `pip install .` or `pip install -e .` for development
|
||||
- **URL:** N/A
|
||||
- **Port:** N/A
|
||||
- **Service:** N/A
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/md to word
|
||||
|
||||
The tool is distributed as a Python package via the `convert` command-line entry point defined in `pyproject.toml`.
|
||||
|
||||
## Environment Variables
|
||||
No environment variables are documented. Configuration is handled via command-line arguments to the `convert` command.
|
||||
|
||||
## Known Issues
|
||||
- Mermaid rendering method not explicitly defined in source files (assumes external tool availability or wrapper library)
|
||||
- No error handling documentation provided
|
||||
- Image embedding strategy and quality settings not documented
|
||||
|
||||
## Git
|
||||
- **Remote:** [Not provided in source files]
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
|
|
@ -25,4 +98,4 @@ Utility for converting Markdown files to Word (.docx). Python-based. No README.
|
|||
## Change Log
|
||||
| Date | Requested | Changed | Files |
|
||||
|------|-----------|---------|-------|
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
|
|
@ -3,8 +3,9 @@ name: "Obsidian Second Brain"
|
|||
client: "Oliver Internal"
|
||||
status: active
|
||||
server: local (iCloud)
|
||||
tech: [Obsidian, Markdown, Dataview, Templater]
|
||||
tech: [Obsidian, Python, Markdown, Dataview, Templater, Claude API]
|
||||
local_path: /Users/ai_leed/Library/Mobile Documents/iCloud~md~obsidian/Documents/VadymSamoilenko
|
||||
scripts_path: /Users/ai_leed/.claude/
|
||||
deploy: "Open Obsidian → vault VadymSamoilenko"
|
||||
url: https://github.com/SamoilenkoVadym/obsidian
|
||||
tags:
|
||||
|
|
@ -15,38 +16,164 @@ created: 2026-04-14
|
|||
---
|
||||
|
||||
## Overview
|
||||
Obsidian-based second brain for all Oliver Agency projects. AI-first knowledge management: Claude Code reads/writes notes via hooks, skills, and CLAUDE.md protocol. Vault mirrors disk structure at `/Users/ai_leed/Documents/Projects/Oliver/`.
|
||||
Obsidian-based second brain for all Oliver Agency projects and personal knowledge. AI-first knowledge management: Claude Code reads/writes notes via hooks at every session start/stop. Project notes are the **source of truth** — tech stack, server, deploy command, URL, changelog, session history. The wiki is auto-populated from sessions and technical work. When starting on any project, Claude reads the project note before asking the user anything.
|
||||
|
||||
## Setup
|
||||
- **Vault:** `/Users/aimpress/Library/Mobile Documents/iCloud~md~obsidian/Documents/VadymSamoilenko`
|
||||
- **Vault:** `~/Library/Mobile Documents/iCloud~md~obsidian/Documents/VadymSamoilenko`
|
||||
- **Projects root:** `/Users/ai_leed/Documents/Projects/Oliver/`
|
||||
- **Git sync:** `github.com/SamoilenkoVadym/obsidian` (private)
|
||||
- **Claude Code skills:** `.agents/skills/` — obsidian-cli, obsidian-markdown, obsidian-bases, json-canvas, defuddle
|
||||
- **Hook scripts:** `~/.claude/obsidian-*.py`
|
||||
- **Memory compiler:** `~/.claude/memory-compiler/` (github.com/coleam00/claude-memory-compiler)
|
||||
- **Claude Code skills:** `~/.agents/skills/` via `kepano/obsidian-skills`
|
||||
|
||||
## Architecture
|
||||
- **PARA structure:** 01 Projects / 02 Areas / 03 Resources / 04 Archive / 05 Aimpress LTD / 99 Daily
|
||||
- **35 project notes** — 1:1 mapping to `/Users/ai_leed/Documents/Projects/Oliver/`
|
||||
- **Dashboards:** `01 Projects/Dashboard.md` (Dataview) + `01 Projects/Projects.base` (Bases)
|
||||
- **Templates:** Daily, Weekly Review, Meeting, Project
|
||||
|
||||
```
|
||||
Claude Code session start
|
||||
→ obsidian-session-start.py
|
||||
→ reads 01 Projects/<project>/*.md
|
||||
→ injects: client, server, tech, overview, recent timeline, last 5 sessions
|
||||
→ memory-compiler/hooks/session-start.py
|
||||
→ injects: wiki/_master-index.md + relevant topic articles
|
||||
|
||||
Work happens...
|
||||
|
||||
Claude Code session stop
|
||||
→ obsidian-session-log.py (Stop hook)
|
||||
→ Haiku: Asked/Done summary + Change Log row
|
||||
→ Haiku: wiki article if significant tech work done
|
||||
→ dedup: skip session entry if today already has 5+
|
||||
→ writes: Sessions, Change Log, Timeline to project note
|
||||
→ writes: 99 Daily/<date>.md entry
|
||||
→ memory-compiler/hooks/session-end.py
|
||||
→ flush.py: knowledge → daily log
|
||||
→ compile.py: daily logs → wiki articles (auto after 21:00)
|
||||
|
||||
Claude Code PostCompact
|
||||
→ obsidian-postcompact.py
|
||||
→ Haiku writes structured session entry to project note
|
||||
|
||||
Manual enrichment (any time):
|
||||
cd /path/to/project
|
||||
PROJECTS_ROOT=... python3 ~/.claude/obsidian-enrich.py
|
||||
```
|
||||
|
||||
## Automation Hooks
|
||||
|
||||
| Hook | Script | Does |
|
||||
|------|--------|------|
|
||||
| SessionStart | `obsidian-session-start.py` | Injects project context + pending commands; auto-creates stub notes; detects stubs |
|
||||
| PostCompact | `obsidian-postcompact.py` | Haiku writes structured session entry on compaction |
|
||||
| Stop | `obsidian-session-log.py` | Timestamps daily note with session duration |
|
||||
| SessionStart | `obsidian-session-start.py` v3 | Injects client, server, tech, overview, recent timeline, last 5 sessions. Auto-creates stub notes. Injects pending commands from `02 Areas/Pending Commands.md` |
|
||||
| SessionStart | `memory-compiler/hooks/session-start.py` | Injects wiki master-index + relevant topic articles |
|
||||
| Stop | `obsidian-session-log.py` | Haiku: session summary + changelog row + wiki article generation. Dedup: max 5 sessions/day per project. Updates project note + daily note |
|
||||
| PostCompact | `obsidian-postcompact.py` | Haiku writes structured session entry to project note on context compaction |
|
||||
| PreCompact | `memory-compiler/hooks/pre-compact.py` | Saves context to knowledge base before compaction |
|
||||
| SessionEnd | `memory-compiler/hooks/session-end.py` | Triggers flush.py → daily log → compile.py → wiki |
|
||||
| PreCompact | `memory-compiler/scripts/wiki-count-sync.py` | Updates article counts in _master-index.md |
|
||||
|
||||
## Key Scripts
|
||||
|
||||
| Script | Purpose |
|
||||
|--------|---------|
|
||||
| `obsidian-session-start.py` | SessionStart hook — injects project context |
|
||||
| `obsidian-session-log.py` | Stop hook — logs session, generates wiki articles |
|
||||
| `obsidian-enrich.py` | **Manual enrichment** — reads project files, generates full note via Haiku |
|
||||
| `obsidian-postcompact.py` | PostCompact hook |
|
||||
| `memory-compiler/scripts/flush.py` | Extracts knowledge from transcripts → daily log |
|
||||
| `memory-compiler/scripts/compile.py` | Compiles daily logs → wiki articles |
|
||||
| `memory-compiler/scripts/query.py` | Query the wiki: `uv run python query.py "question" [--file-back]` |
|
||||
|
||||
## Vault Structure (PARA)
|
||||
|
||||
```
|
||||
VadymSamoilenko/
|
||||
├── 01 Projects/ ← project notes (source of truth)
|
||||
│ ├── <project-name>/ ← 1:1 with /Documents/Projects/Oliver/
|
||||
│ │ └── *.md ← overview, tech, server, deploy, sessions, changelog
|
||||
│ ├── Dashboard.md ← Dataview dashboard (5 views)
|
||||
│ └── Projects.base ← Bases database view
|
||||
├── 02 Areas/
|
||||
│ └── Pending Commands.md ← injected at every session start
|
||||
├── 03 Resources/
|
||||
├── 04 Archive/
|
||||
├── 05 Aimpress LTD/
|
||||
├── 99 Daily/ ← auto-written by Stop hook
|
||||
└── wiki/ ← LLM's domain — auto-written
|
||||
├── _master-index.md ← master index, injected at session start
|
||||
├── concepts/ ← 62+ articles
|
||||
├── tech-patterns/ ← how-to: Docker, FastAPI, Azure AD, etc.
|
||||
├── architecture/ ← architectural decisions + patterns
|
||||
├── connections/ ← cross-cutting insights
|
||||
├── homelab/ ← 38 articles
|
||||
├── agent-sdk/ ← 30 articles
|
||||
└── ... ← 15+ topic folders
|
||||
```
|
||||
|
||||
## Project Note Standard
|
||||
|
||||
Every project note must have:
|
||||
- **Frontmatter:** name, client, status, tech, server, deploy, url, port, db, service
|
||||
- **## Overview** — what it is, who uses it, key capabilities
|
||||
- **## Tech Stack** — frontend, backend, db, infrastructure, AI/ML
|
||||
- **## Architecture** — components, data flow, diagram
|
||||
- **## Dev Commands** — exact commands to run locally
|
||||
- **## Deployment** — server, deploy command, URL, port, service name
|
||||
- **## Environment Variables** — key env vars
|
||||
- **## Sessions** — auto-written by Stop hook (max 5/day)
|
||||
- **## Change Log** — auto-written by Stop hook (every session)
|
||||
- **## Timeline** — auto-written from git commits
|
||||
|
||||
To enrich any incomplete note:
|
||||
```bash
|
||||
cd /path/to/project
|
||||
PROJECTS_ROOT=/Users/ai_leed/Documents/Projects/Oliver python3 ~/.claude/obsidian-enrich.py
|
||||
```
|
||||
|
||||
## Installed Skills (kepano/obsidian-skills)
|
||||
- `obsidian-cli` — CLI read/write (requires Obsidian open + CLI enabled)
|
||||
- `obsidian-markdown` — Obsidian Flavored Markdown syntax
|
||||
- `obsidian-bases` — .base file creation
|
||||
- `json-canvas` — Canvas files
|
||||
- `defuddle` — Clean markdown from web pages
|
||||
|
||||
From **Steph Yin (kepano)**, creator of Obsidian: `github.com/kepano/obsidian-skills`
|
||||
|
||||
| Skill | Trigger | Does |
|
||||
|-------|---------|------|
|
||||
| `obsidian:obsidian-cli` | "interact with vault", "search notes" | CLI read/write — requires Obsidian open + CLI enabled |
|
||||
| `obsidian:obsidian-markdown` | Working with .md in Obsidian | Wikilinks, callouts, embeds, frontmatter syntax |
|
||||
| `obsidian:obsidian-bases` | ".base files", "database view" | Create Bases views (.base files) |
|
||||
| `obsidian:json-canvas` | "canvas", "diagram" | Create .canvas files |
|
||||
| `obsidian:defuddle` | "read this URL", web pages | Extracts clean markdown from web (saves tokens vs WebFetch) |
|
||||
|
||||
Install more: `/plugin marketplace add kepano/obsidian-skills`
|
||||
|
||||
## Plugins
|
||||
calendar, obsidian-git, obsidian-kanban, templater-obsidian, obsidian-tasks-plugin, dataview, nldates-obsidian, obsidian-outliner, mermaid-tools, obsidian-html-plugin, omnisearch
|
||||
|
||||
## CLAUDE.md Rules (global)
|
||||
|
||||
Claude Code is configured (in `~/.claude/CLAUDE.md`) to:
|
||||
1. Search Obsidian vault before asking user for project info
|
||||
2. Read `01 Projects/<project>/*.md` for server, URL, tech, deploy command
|
||||
3. Read `wiki/_master-index.md` → topic index → articles
|
||||
4. Run `obsidian-enrich.py` when starting on a stub project
|
||||
5. Ask user only if info is genuinely missing
|
||||
|
||||
## Wiki Source of Truth
|
||||
|
||||
Wiki articles come from:
|
||||
1. **flush.py** → sessions transcripts → `concepts/`, `tech-patterns/`, etc.
|
||||
2. **obsidian-session-log.py** → significant tech work detected → article written to wiki
|
||||
3. **Manual compile:** `cd ~/.claude/memory-compiler && uv run python scripts/compile.py`
|
||||
4. **Query + file-back:** `uv run python scripts/query.py "question" --file-back`
|
||||
|
||||
## Sessions
|
||||
|
||||
### 2026-04-29 — Second Brain System Overhaul
|
||||
**Asked:** Make Obsidian second brain the source of truth; wiki auto-populated; Claude searches Obsidian first; complete project notes.
|
||||
**Done:**
|
||||
- `obsidian-session-start.py` v3: now injects client, server, tech, overview, recent timeline, last 5 sessions (by count not lines); `extract_frontmatter` parses tech list; `extract_last_sessions` replaces `extract_sessions_section`
|
||||
- `obsidian-session-log.py`: added session deduplication (max 5/day per project), added wiki article generation (second Haiku call — writes to wiki/tech-patterns or wiki/concepts if significant work detected)
|
||||
- `obsidian-enrich.py`: NEW script — reads CLAUDE.md, README, package.json, docker-compose.yml, git log, generates complete project note via Haiku; preserves Sessions/Changelog/Timeline
|
||||
- `~/.claude/CLAUDE.md`: added "Second Brain" section — Claude now instructed to search Obsidian before asking user
|
||||
- Mass enrichment: ran `obsidian-enrich.py` on all 28 stub project notes
|
||||
- Confirmed kepano/obsidian-skills installed (5 skills from Obsidian creator)
|
||||
|
||||
### 2026-04-27 — Wiki Knowledge Base Expansion
|
||||
**Requested:** Расширить wiki чтобы максимально помогать в разработке; автоматически обновлять index файлы
|
||||
**Done:**
|
||||
|
|
@ -54,35 +181,29 @@ calendar, obsidian-git, obsidian-kanban, templater-obsidian, obsidian-tasks-plug
|
|||
- 3 новые client-knowledge статьи: barclays, ferrero, 3m
|
||||
- 5 новых connections статей: fastapi-azuread-docker-trinity, ai-always-needs-cost-tracker, optical-dev-apache-vite-basepath, gcp-no-websockets, box-api-hotfolder-pattern
|
||||
- Обновлён CLAUDE.md: добавлен Q&A Auto-Save Protocol
|
||||
- Обновлён Barclays Banner Builder project note (был stub)
|
||||
- `wiki-count-sync.py` расширен до полного `wiki-index-sync` — теперь автоматически добавляет строки в `_index.md` при новых статьях + обновляет счётчики в `_master-index.md`; запускается в PreCompact хуке
|
||||
- `wiki-count-sync.py` расширен до полного `wiki-index-sync`
|
||||
|
||||
### 2026-04-14 — Full Setup + Improvements
|
||||
**Requested:** Analyse vault and implement improvements, enrichment, automation
|
||||
**Done:**
|
||||
- Analysed vault structure, hooks, templates
|
||||
- Reviewed kepano/obsidian-skills and kepano/kepano-obsidian for best practices
|
||||
- Fixed PostCompact Change Log parser bug (regex mismatch)
|
||||
- Fixed date format inconsistency (DD-MM-YYYY → YYYY-MM-DD in configs + renamed 5 daily notes)
|
||||
- Set up Git sync: `SamoilenkoVadym/obsidian` (private, 9 commits)
|
||||
- Installed 5 obsidian-skills (cli, markdown, bases, canvas, defuddle)
|
||||
- Structural cleanup: moved 9 loose notes from 01 Projects/ → 03 Resources/Infrastructure/ + SOPs/
|
||||
- Deleted empty Untitled.canvas and Untitled.base
|
||||
- Created: Work Tasks.md (Dataview), Infrastructure Index.md, Dashboard.md (5 views), Projects.base (4 views)
|
||||
- New templates: Weekly Review (fractal journaling), Daily Template upgraded
|
||||
- SessionStart hook v2: auto-create notes for new projects + stub detection
|
||||
- Stop hook: session duration tracking
|
||||
- CLAUDE.md: tag taxonomy, status lifecycle, archival workflow, obsidian-skills reference
|
||||
- Templater folder templates configured
|
||||
- Full vault analysis, hook setup, kepano skills install
|
||||
- Fixed PostCompact parser bug, date format inconsistency
|
||||
- Git sync setup: `SamoilenkoVadym/obsidian`
|
||||
- Dashboard.md + Projects.base created
|
||||
- SessionStart v2: auto-create stubs + stub detection
|
||||
- CLAUDE.md: tag taxonomy, status lifecycle, archival workflow
|
||||
|
||||
---
|
||||
|
||||
## Change Log
|
||||
| Date | Requested | Changed | Files |
|
||||
|------|-----------|---------|-------|
|
||||
| 2026-04-14 | Full vault improvement session | Setup, fixes, automation, dashboards, templates | 20+ files |
|
||||
| 2026-04-29 | Second brain system overhaul | session-start v3, session-log dedup+wiki, obsidian-enrich.py, CLAUDE.md, mass enrichment 28 projects | obsidian-session-start.py, obsidian-session-log.py, obsidian-enrich.py, CLAUDE.md |
|
||||
| 2026-04-27 | Wiki expansion + auto-indexing | 11 new wiki articles, wiki-index-sync, CLAUDE.md | wiki/*, memory-compiler/scripts |
|
||||
| 2026-04-14 | Full vault setup | Hooks, skills, templates, git sync, dashboards | 20+ files |
|
||||
|
||||
## Related
|
||||
- [[Projects Index]]
|
||||
- [[Dashboard]]
|
||||
- [[CLAUDE]]
|
||||
- [[wiki/obsidian-rag/karpathy-llm-wiki-approach|Karpathy LLM Wiki Approach]]
|
||||
- [[wiki/obsidian-rag/claude-code-obsidian-setup|Claude Code + Obsidian Setup]]
|
||||
|
|
|
|||
|
|
@ -1,33 +1,185 @@
|
|||
---
|
||||
name: "OliVAS — Visual Attention Software"
|
||||
client: Oliver Internal
|
||||
client: OLIVER
|
||||
status: active
|
||||
server: optical-web-1
|
||||
tech: [Python, FastAPI, Docker, Claude API, DeepGaze]
|
||||
server: optical-dev
|
||||
tech: [React 18, TypeScript, FastAPI, Python 3.12, PostgreSQL, DeepGaze, Claude Sonnet, Docker]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/olivas
|
||||
deploy: docker compose up --build
|
||||
url:
|
||||
url: https://optical-dev.oliver.solutions/olivas
|
||||
tags: [oliver, visual-attention, ai, heatmap, saliency, design-analysis]
|
||||
created: 2026-04-14
|
||||
port: 1577
|
||||
db: PostgreSQL 16
|
||||
---
|
||||
|
||||
## Overview
|
||||
Open-source Visual Attention Suite — predicts where humans look in an image during first 3-5 seconds. Built for creative teams at OLIVER. No physical eye-tracking needed.
|
||||
|
||||
Features: saliency heatmap, gaze sequence (numbered fixation points), hotspot detection (top 5), attention score (0-100), Areas of Interest (AOI), rule-based insights, Claude Sonnet 4.6 AI design analysis, PDF reports.
|
||||
|
||||
Models: DeepGaze I, DeepGaze IIE (recommended), DeepGaze III.
|
||||
**OliVAS** (OLIVER Visual Attention Suite) is an open-source web application that predicts where humans will look in an image during the first 3–5 seconds of viewing, without requiring physical eye-tracking hardware. Built for creative teams, designers, and marketers at OLIVER, it delivers saliency heatmaps, gaze sequence predictions, hotspot analysis, and AI-powered design insights. Users can analyze images via deep learning models (DeepGaze I/IIE/III), draw Areas of Interest (AOI) for detailed attention measurement, and generate professional PDF reports with both rule-based and optional Claude Sonnet–powered recommendations.
|
||||
|
||||
## Tech Stack
|
||||
- **Backend:** Python + FastAPI
|
||||
- **Vision:** DeepGaze I/IIE/III
|
||||
- **AI Analysis:** Claude Sonnet 4.6
|
||||
- **Infrastructure:** Docker + docker-compose
|
||||
|
||||
- **Frontend:** React 18, TypeScript, Vite, Tailwind CSS, Zustand (state), React Router
|
||||
- **Backend:** FastAPI, Python 3.12, SQLAlchemy (async ORM), Pydantic v2
|
||||
- **Database:** PostgreSQL 16 (async driver: asyncpg)
|
||||
- **Infrastructure:** Docker Compose, deployed to Google Cloud Run (optional offloading), Azure AD SSO
|
||||
- **AI/ML:** DeepGaze I/IIE/III (saliency models), Anthropic Claude Sonnet 4.6 (optional design analysis), OpenAI CLIP (optional)
|
||||
- **Key Libraries:**
|
||||
- `deepgaze-pytorch` — visual attention prediction
|
||||
- `ReportLab` — PDF generation with Montserrat typography
|
||||
- `Alembic` — database migrations
|
||||
- `MSAL` — Azure AD authentication
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ User Browser │
|
||||
│ React SPA (Vite) — /olivas subpath, Azure AD SSO login │
|
||||
└────────────────────────┬────────────────────────────────────┘
|
||||
│ HTTPS (CORS: optical-dev.oliver.solutions)
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ FastAPI Backend (8000) │
|
||||
│ ┌─────────────┐ ┌──────────────┐ ┌─────────────────┐ │
|
||||
│ │Auth (Azure) │ │Analysis API │ │Project Mgmt │ │
|
||||
│ └─────────────┘ └──────────────┘ └─────────────────┘ │
|
||||
│ │ │ Saliency │ CRUD │
|
||||
│ └──────────────┼─────────────────┴──────────────┐ │
|
||||
│ ▼ │ │
|
||||
│ ┌─────────────────────────────┐ │ │
|
||||
│ │ ML Processing Service │ │ │
|
||||
│ │ • DeepGaze (CPU/GPU) │ │ │
|
||||
│ │ • Image preprocessing │ │ │
|
||||
│ │ • PDF generation (ReportLab)│ │ │
|
||||
│ │ • Optional: Cloud Run offload│ │ │
|
||||
│ └─────────────────────────────┘ │ │
|
||||
│ │ │ │
|
||||
└────────────────────────┼────────────────────────────────┼───┘
|
||||
│ │
|
||||
┌────────────────┼────────────────┐ │
|
||||
▼ ▼ ▼ │
|
||||
┌─────────┐ ┌──────────────┐ ┌──────────┐ │
|
||||
│ Local │ │ PostgreSQL │ │File │ │
|
||||
│GPU/CPU │ │(5432→5453) │ │Storage │ │
|
||||
└─────────┘ │ (olivas DB) │ │/uploads │ │
|
||||
└──────────────┘ └──────────┘ │
|
||||
│
|
||||
┌───────────────────────────────────────────────────┐ │
|
||||
│ Optional: Google Cloud Run Saliency Service │ │
|
||||
│ (CLOUD_RUN_SALIENCY_URL env var) │ │
|
||||
└───────────────────────────────────────────────────┘ │
|
||||
▲ │
|
||||
└────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Data Flow:**
|
||||
1. User logs in via Azure AD (MSAL on frontend)
|
||||
2. Uploads image → backend stores in `/app/data/uploads`
|
||||
3. Backend queues saliency analysis (local GPU/CPU or Cloud Run)
|
||||
4. DeepGaze model generates attention heatmap + fixation points
|
||||
5. Backend optionally calls Claude Sonnet for design insights
|
||||
6. Frontend displays interactive heatmap, hotspots, AOI tools
|
||||
7. User generates PDF report → ReportLab merges visuals + metrics + insights
|
||||
8. Reports stored in PostgreSQL and file system
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
# Clone and navigate
|
||||
git clone git@bitbucket.org:zlalani/olivas.git
|
||||
cd olivas
|
||||
|
||||
# Full setup (Python 3.12, Node 18 required)
|
||||
make setup
|
||||
|
||||
# Start database
|
||||
make db-up
|
||||
|
||||
# Run migrations
|
||||
make db-migrate
|
||||
|
||||
# Start all three services in separate terminals
|
||||
make dev-backend # Backend on 8000
|
||||
make dev-frontend # Frontend on 1577
|
||||
|
||||
# Or use single command (runs in background)
|
||||
make dev
|
||||
|
||||
# Docker deployment
|
||||
make docker-up # Full stack via docker-compose
|
||||
make docker-down # Stop stack
|
||||
|
||||
# Testing & linting
|
||||
make test
|
||||
make lint
|
||||
make lint-fix
|
||||
|
||||
# Clean build artifacts
|
||||
make clean
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `docker compose up --build`
|
||||
|
||||
- **Server:** optical-dev (subpath `/olivas`)
|
||||
- **Deploy:** `docker compose up --build` (or see production deploy script)
|
||||
- **URL:** https://optical-dev.oliver.solutions/olivas
|
||||
- **Frontend Port:** 1577 (internal); served via Apache reverse proxy on :443
|
||||
- **Backend Port:** 8000
|
||||
- **Service:** Docker Compose (no systemd service)
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/olivas`
|
||||
|
||||
**Cloud Offloading (optional):**
|
||||
- Saliency inference can offload to Google Cloud Run via `CLOUD_RUN_SALIENCY_URL` and `CLOUD_RUN_PROCESSING_URL`
|
||||
- Requires `GOOGLE_CLOUD_PROJECT` and `CLOUD_RUN_SECRET` env vars
|
||||
|
||||
## Environment Variables
|
||||
|
||||
### Backend (`.env`)
|
||||
- `DATABASE_URL` — PostgreSQL connection string (e.g., `postgresql+asyncpg://olivas:olivas@localhost:5453/olivas`)
|
||||
- `UPLOAD_DIR` — Path to store uploaded images (default: `./data/uploads`)
|
||||
- `DEVICE` — ML inference device: `auto` | `cpu` | `cuda` (default: `auto`)
|
||||
- `BACKEND_HOST` — Bind address (default: `0.0.0.0`)
|
||||
- `BACKEND_PORT` — Port (default: `8000`)
|
||||
- `CORS_ORIGINS` — Comma-separated CORS whitelist
|
||||
- `ANTHROPIC_API_KEY` — Claude API key (optional; leave empty to disable AI insights)
|
||||
- `AZURE_AUTH_ENABLED` — Enable Azure AD SSO (default: `true`)
|
||||
- `AZURE_TENANT_ID` — Azure tenant ID
|
||||
- `AZURE_CLIENT_ID` — Azure app registration client ID
|
||||
- `CLOUD_RUN_SALIENCY_URL` — Google Cloud Run endpoint for saliency (optional)
|
||||
- `CLOUD_RUN_PROCESSING_URL` — Google Cloud Run endpoint for image processing (optional)
|
||||
- `CLOUD_RUN_SECRET` — Secret for Cloud Run authentication (optional)
|
||||
- `GOOGLE_CLOUD_PROJECT` — GCP project ID (default: `optical-414516`)
|
||||
|
||||
### Frontend (Vite env vars in `.env`)
|
||||
- `VITE_AZURE_TENANT_ID` — Azure tenant ID
|
||||
- `VITE_AZURE_CLIENT_ID` — Azure app registration client ID
|
||||
- `VITE_AZURE_REDIRECT_URI` — OAuth2 redirect URI (e.g., `https://optical-dev.oliver.solutions/olivas`)
|
||||
|
||||
### Docker Compose
|
||||
- All above env vars can be injected into `docker-compose.yml` via shell environment or `.env` file
|
||||
|
||||
## API / Endpoints
|
||||
|
||||
Backend (FastAPI) exposes REST endpoints. Key areas (not exhaustively documented in source):
|
||||
- **Auth:** Azure AD MSAL integration — token validation on all protected routes
|
||||
- **Images:** `POST /api/images/upload` → store and return metadata
|
||||
- **Analyses:** `POST /api/analyses/saliency` → trigger DeepGaze model
|
||||
- **Hotspots:** `GET /api/analyses/{id}/hotspots` → top 5 attention regions
|
||||
- **AOI:** `POST /api/aoi/create` → define Areas of Interest, measure attention
|
||||
- **Projects:** CRUD operations for organizing analyses
|
||||
- **Reports:** `GET /api/reports/{id}` → download PDF
|
||||
- **Comparison:** Side-by-side analysis comparison endpoint
|
||||
|
||||
(Full OpenAPI docs available at `/docs` when backend is running)
|
||||
|
||||
## Known Issues
|
||||
|
||||
- **Frontend auth:** Images must be loaded via authenticated fetch (not bare `<img src>`) due to CORS and SSO — recent commits (e.g., `e9e19fb`) fixed this
|
||||
- **Subpath deployment:** Requires careful Apache rewrite configuration and React Router basename — see commits `91a0104`, `8484aab`
|
||||
- **Database migrations:** Alembic PYTHONPATH must be set correctly in production (`c6cd329`)
|
||||
- **Cloud Run offloading:** Not required but optional for scaling — incomplete in documentation
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
**Done:** Added to Obsidian second brain.
|
||||
|
|
@ -40,4 +192,4 @@ Models: DeepGaze I, DeepGaze IIE (recommended), DeepGaze III.
|
|||
|
||||
## Related
|
||||
- [[cinema-studio-pro/Cinema Studio Pro]]
|
||||
- [[semblance/Semblance]]
|
||||
- [[semblance/Semblance]]
|
||||
|
|
@ -1,27 +1,177 @@
|
|||
---
|
||||
name: "Oliver Sales Ops Platform"
|
||||
client: "TBD"
|
||||
client: OLIVER
|
||||
status: active
|
||||
tech: []
|
||||
tech: [Python, FastAPI, React, TypeScript, PostgreSQL, Redis, Claude API, Docker]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/oliver-sales-ops-platform
|
||||
deploy:
|
||||
url:
|
||||
server:
|
||||
deploy: ./deploy.sh
|
||||
url: http://optical-dev/oliver-sales-ops-platform
|
||||
server: optical-dev
|
||||
tags:
|
||||
- project
|
||||
created: 2026-04-28
|
||||
port: 8003
|
||||
db: PostgreSQL
|
||||
---
|
||||
|
||||
## Overview
|
||||
> New project — fill in during first session.
|
||||
|
||||
**oliver-sales-ops-platform** is an end-to-end RFP-to-mobilization pipeline for OLIVER's commercial team. It automates the 17-stage process of transforming a brief into a defensible proposal: intake, qualification, asset matching, ratecard generation, delivery model recommendation, team shaping, validation, and pitch material generation. Users drop in a brief and walk it through stages that combine Claude AI agents, human decisions, and pure-Python calculations to produce structured proposals with full cost tracking.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:**
|
||||
- **Backend:**
|
||||
- **Infrastructure:**
|
||||
|
||||
- **Frontend:** React 18, TypeScript, Vite, TailwindCSS
|
||||
- **Backend:** FastAPI (Python 3.11+), SQLAlchemy ORM, Alembic migrations
|
||||
- **Database:** PostgreSQL 16 + Redis 7 (for sessions/queuing)
|
||||
- **Infrastructure:** Docker Compose, Apache (prod), systemd (optional)
|
||||
- **AI/ML:** Anthropic Claude API (multiple specialized agents per stage)
|
||||
- **Key libraries:** `anthropic`, `pydantic`, `asyncpg`, `mailgun-python`, Azure AD/MSAL
|
||||
|
||||
## Architecture
|
||||
|
||||
The platform follows a **stage-machine** pattern: each Opportunity moves through 17 immutable stages. Stages are strictly ordered (`/stages/{n}/complete` enforces progression). The flow is:
|
||||
|
||||
```
|
||||
Brief (Intake)
|
||||
↓ (Claude Diagnosis Agent)
|
||||
↓ (Human Qualification + Approval Gate)
|
||||
↓ (Asset Matching + Ratecard Calculation)
|
||||
↓ (Delivery Model + Team Shape + Gaps)
|
||||
↓ (Human Efficiency Tuning + Approval Gate)
|
||||
↓ (Pitch Materials + Post-Win Planning)
|
||||
→ [Phase 2: Salesforce/SharePoint push deferred]
|
||||
```
|
||||
|
||||
**Key design decisions:**
|
||||
|
||||
1. **Immutable stage artifacts:** Each stage produces a JSON `stage_artifact` record; artifacts are never overwritten, only new ones created.
|
||||
2. **Approval gates:** Stages 3 and 14 require explicit `Approval` rows (human review/sign-off). Approvals fan out via in-app notifications and Mailgun email.
|
||||
3. **Cost tracking:** Every Claude call records tokens (in/out) and USD cost on the artifact; per-stage spend is visible in the UI.
|
||||
4. **Pure-Python calculations:** Ratecard (stage 8) and Team Shape (stage 11) use deterministic Python logic, not AI.
|
||||
5. **Async-first backend:** FastAPI with `asyncpg` for I/O; Redis for caching and session management.
|
||||
6. **Dev auth bypass:** For local dev, `DEV_AUTH_BYPASS=true` skips Azure AD; production uses Azure AD + allowlist (`allowed_users.yaml`).
|
||||
|
||||
**Data model:**
|
||||
- `Opportunity` (the brief/project container)
|
||||
- `Stage` (immutable records: stage number, status, timestamps)
|
||||
- `StageArtifact` (JSON output from each stage, includes cost data)
|
||||
- `Approval` (sign-off records for gated stages)
|
||||
- `User` (SSO identity, role-based access)
|
||||
- `Notification` (in-app alerts; Mailgun for email)
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
# Clone and setup
|
||||
git clone git@bitbucket.org:zlalani/oliver-sales-ops-platform.git
|
||||
cd oliver-sales-ops-platform
|
||||
cp .env.example .env
|
||||
# Edit .env: set ANTHROPIC_API_KEY, optional AZURE_* for SSO, MAILGUN_* for emails
|
||||
$EDITOR .env
|
||||
|
||||
# Start all services (db, redis, backend, frontend)
|
||||
docker compose up -d
|
||||
|
||||
# Frontend only: Vite dev server (for hot-reload)
|
||||
COMPOSE_PROFILES=dev docker compose up -d frontend
|
||||
|
||||
# Backend logs
|
||||
docker compose logs -f backend
|
||||
|
||||
# Database migrations (run inside backend container or via CLI)
|
||||
docker compose exec backend alembic upgrade head
|
||||
|
||||
# Run tests (pytest — backend only)
|
||||
docker compose exec backend pytest tests/ -v
|
||||
|
||||
# Stop all services
|
||||
docker compose down
|
||||
|
||||
# Rebuild after code changes (e.g., new dependencies)
|
||||
docker compose build backend
|
||||
docker compose up -d backend
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Local path:** `/Volumes/SSD/Projects/Oliver/oliver-sales-ops-platform`
|
||||
|
||||
- **Server:** `optical-dev` (Apache + systemd)
|
||||
- **Deploy script:** `./deploy.sh` (builds SPA static files, syncs to server, restarts service)
|
||||
- **URL:** `http://optical-dev/oliver-sales-ops-platform` (or per `APP_PUBLIC_URL`)
|
||||
- **Backend port:** 8003 (internal Docker; proxied via Apache)
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/oliver-sales-ops-platform`
|
||||
|
||||
**Deploy procedure:**
|
||||
```bash
|
||||
./deploy.sh
|
||||
```
|
||||
This script:
|
||||
1. Builds the React SPA (`npm run build` in `frontend/`)
|
||||
2. Finds free ports for backend/frontend (or uses configured ones)
|
||||
3. Renders Apache config from template (`deploy/apache-conf.tpl`)
|
||||
4. Syncs code + Docker images to `optical-dev`
|
||||
5. Restarts backend service (systemd or docker-compose on remote)
|
||||
|
||||
## Environment Variables
|
||||
|
||||
**Backend (`backend/.env` or docker-compose):**
|
||||
- `DATABASE_URL` — PostgreSQL async connection string (asyncpg)
|
||||
- `DATABASE_URL_SYNC` — PostgreSQL sync string (Alembic migrations)
|
||||
- `REDIS_URL` — Redis connection (default: `redis://redis:6379/0`)
|
||||
- `ANTHROPIC_API_KEY` — Claude API key (required)
|
||||
- `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` — Azure AD SSO (leave blank to disable)
|
||||
- `DEV_AUTH_BYPASS` — Set to `true` for local dev to skip SSO (NEVER in production)
|
||||
- `DEV_AUTH_EMAIL`, `DEV_AUTH_NAME`, `DEV_AUTH_ROLE` — Dev user identity (if `DEV_AUTH_BYPASS=true`)
|
||||
- `MAILGUN_API_KEY`, `MAILGUN_DOMAIN`, `MAILGUN_FROM`, `MAILGUN_REGION` — Email config (leave key blank in dev to log instead)
|
||||
- `APP_PUBLIC_URL` — Base URL for approval email links (e.g., `http://localhost:3011`)
|
||||
- `APP_PATH_PREFIX` — URL path prefix (e.g., `/oliver-sales-ops-platform`)
|
||||
- `ALLOWED_USERS_PATH` — Path to `allowed_users.yaml` (default: `/app/config/allowed_users.yaml`)
|
||||
- `DATA_DIR` — Path to reference data (GMAL Excel, etc.; default: `./data`)
|
||||
|
||||
**Frontend (`frontend/.env` or Vite config):**
|
||||
- `VITE_DEV_AUTH_BYPASS` — Same as backend; must match for dev auth to work
|
||||
|
||||
## API / Endpoints
|
||||
|
||||
**Key REST routes (backend):**
|
||||
- `GET /api/opportunities` — List all opportunities
|
||||
- `POST /api/opportunities` — Create new opportunity
|
||||
- `GET /api/opportunities/{id}` — Get opportunity + stages
|
||||
- `POST /api/opportunities/{id}/stages/{stage_num}/complete` — Advance to next stage
|
||||
- `GET /api/opportunities/{id}/stages/{stage_num}/artifact` — Fetch stage output
|
||||
- `POST /api/approvals` — Submit approval (gated stages)
|
||||
- `GET /api/approvals?opportunity_id={id}` — List approvals for opportunity
|
||||
- `GET /api/me` — Current user info
|
||||
- `GET /api/health` — Health check
|
||||
|
||||
**WebSocket (if applicable):**
|
||||
- Real-time notifications (approval updates, stage completion)
|
||||
|
||||
## The 17 Stages (Reference)
|
||||
|
||||
| # | Stage | AI Agent | Output | Gated? |
|
||||
|---|-------|----------|--------|--------|
|
||||
| 1 | Intake | Intake Agent | Opportunity metadata (scope, team, budget) | No |
|
||||
| 2 | Read & Diagnose Brief | Diagnosis Agent | Key findings, clarifications, risks | No |
|
||||
| 3 | **Qualification Assessment** | Human + TROWLS form | Qualification score | **Yes** |
|
||||
| 4 | Generate Client Q&A Pack | Pure-Python | Excel/Word export | No |
|
||||
| 5 | Ingest Client Answers | Human edit | Updated JSON | No |
|
||||
| 6 | Normalize Asset List | Asset Normalizer | Standardized asset taxonomy | No |
|
||||
| 7 | Match Assets to Job Routes | Match Agent | Asset-to-route mappings + hours | No |
|
||||
| 8 | Build Asset-Level Rate Card | Pure-Python (GMAL) | Hourly rates × volume → cost | No |
|
||||
| 9 | Recommend Delivery Model | Delivery Model Agent | Model choice (FTE, SOW, hybrid) + rationale | No |
|
||||
| 10 | Apply Efficiency Logic | Human (UI sliders) | Efficiency profile (cost savings %) | No |
|
||||
| 11 | Create Draft Team Shape | Pure-Python (FTE calc) | Org chart, roles, headcount | No |
|
||||
| 12 | Identify Capability Gaps | Capability Gap Agent | Gaps + training/hiring recommendations | No |
|
||||
| 13 | Generate Support Docs | Support Docs Agent | SLAs, KPIs, governance, caveats | No |
|
||||
| 14 | **Validation & Approval Gates** | Human review | Final sign-off | **Yes** |
|
||||
| 15 | Build Pitch Materials | Pitch Deck Agent | Markdown + Claude-generated slides | No |
|
||||
| 16 | Delivery Planning (post-win) | Implementation Plan Agent | Timeline, milestones, dependencies | No |
|
||||
| 17 | Trigger Downstream Systems | ⏳ Phase 2 | Push to Salesforce / SharePoint | N/A |
|
||||
|
||||
## Known Issues
|
||||
|
||||
- **Stage 17 deferred:** Salesforce/SharePoint integration planned for Phase 2.
|
||||
- **Email in dev:** Set `MAILGUN_API_KEY=""` to log
|
||||
|
||||
## Sessions
|
||||
### 2026-04-29 – Add SPA redirect URI to Azure
|
||||
|
|
@ -124,4 +274,4 @@ created: 2026-04-28
|
|||
| 2026-04-28 | SSO access control | Created allowlist service, added auth middleware with email verification, configured redirect URL | config/allowed_users.yaml, backend/app/services/allowlist.py, backend/app/middleware/auth.py |
|
||||
| 2026-04-28 | SSO user restrictions | AuthProvider logic review, _upsert_app_user docstring update | AuthProvider.tsx, _upsert_app_user |
|
||||
|
||||
## Related
|
||||
## Related
|
||||
|
|
@ -1,79 +1,182 @@
|
|||
---
|
||||
name: "PDF Accessibility Checker"
|
||||
client: Oliver Internal
|
||||
client: Oliver
|
||||
status: active
|
||||
tech: [Python, PHP, PostgreSQL, Redis, Docker, Claude, Google Cloud Vision, pypdf, pdfplumber]
|
||||
tech: [Python, PHP, JavaScript, PostgreSQL, Redis, Docker, Anthropic Claude, Google Cloud Vision]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/pdf-accessibility
|
||||
deploy: docker-compose up (dev) / docker-compose -f docker-compose.prod.yml up -d (prod)
|
||||
url:
|
||||
deploy: docker-compose up -d (production: docker-compose -f docker-compose.prod.yml up -d)
|
||||
url: https://ai-sandbox.oliver.solutions/pdf-accessibility
|
||||
server: optical-web-1
|
||||
tags: [oliver, pdf, accessibility, wcag, ai, php, redis, postgresql]
|
||||
created: 2026-04-14
|
||||
last_commit: 2026-03-18
|
||||
commits: 69
|
||||
port: 8000
|
||||
db: PostgreSQL
|
||||
---
|
||||
|
||||
## Overview
|
||||
Production-ready PDF accessibility checker — WCAG 2.1 Level A & AA validation. ~95% automated coverage via traditional PDF analysis + AI. Branded for Oliver (Montserrat font, black/#FFC407).
|
||||
|
||||
Recent (Feb 2026): API authentication added, production readiness enhancements.
|
||||
pdf-accessibility is an **AI-powered PDF accessibility validation system** that checks documents against WCAG 2.1 Level A & AA standards, achieving ~95% automated coverage. It combines traditional PDF analysis (pypdf, pdfplumber) with cutting-edge AI models (Anthropic Claude 3.5 Sonnet, Google Cloud Vision) to validate accessibility across 30+ criteria. The system serves enterprise users via a modern web UI with drag-and-drop uploads, a RESTful API with authentication, and a CLI tool for batch processing. Branded for Oliver with Montserrat font and black/#FFC407 color palette.
|
||||
|
||||
## Tech Stack
|
||||
- **Web UI:** Vanilla JS + HTML/CSS (drag-drop, visual inspector)
|
||||
- **REST API:** PHP (`api.php`) + `auth.php` (Bearer/X-API-Key)
|
||||
- **Core engine:** Python (`enterprise_pdf_checker.py`) — 30+ WCAG checks
|
||||
- **AI Analysis:** Anthropic Claude + Google Cloud Vision (image analysis)
|
||||
- **PDF libs:** pypdf + pdfplumber
|
||||
- **Queue:** Redis (`pdf:queue`)
|
||||
- **Database:** PostgreSQL (job tracking + history)
|
||||
- **Worker:** `worker.py` (background daemon — Redis queue consumer)
|
||||
- **Infrastructure:** Docker + docker-compose (dev + prod variants)
|
||||
- **Remediation:** `pdf_remediation.py` (auto-fix output)
|
||||
- **Frontend:** Vanilla JavaScript, HTML5/CSS3 (drag-drop file upload, visual inspector, dark mode, responsive design)
|
||||
- **Backend:** Python 3 (core engine), PHP (REST API and authentication layer)
|
||||
- **Database:** PostgreSQL 16 (job tracking, audit logging, results storage)
|
||||
- **Infrastructure:** Docker Compose (development & production stacks), Redis (job queue), Structured logging with rotation
|
||||
- **AI/ML:** Anthropic Claude 3.5 Sonnet (image alt text validation), Google Cloud Vision (OCR, text-in-images detection)
|
||||
- **Key libraries:** pypdf (PDF parsing), pdfplumber (text extraction), pdf2image (rasterization), Pillow (image processing), pytesseract (OCR), textblob (readability), weasyprint (PDF report generation), veraPDF (PDF/UA-1 validation)
|
||||
|
||||
## Architecture
|
||||
|
||||
The system uses a **three-interface architecture** with a centralized asynchronous job queue backend:
|
||||
|
||||
```
|
||||
Web UI / REST API (PHP)
|
||||
↓ auth.php validates Bearer/X-API-Key
|
||||
api.php → uploads/ → Redis queue (pdf:queue)
|
||||
↓
|
||||
worker.py (daemon)
|
||||
└── EnterprisePDFChecker.check_all() → 30+ WCAG checks
|
||||
├── Traditional: pypdf, pdfplumber
|
||||
└── AI: Claude + Google Cloud Vision
|
||||
→ results/{job_id}.result.json + PostgreSQL
|
||||
↓
|
||||
Client polls api.php?action=status → fetch results
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Three User Interfaces │
|
||||
├─────────────────────┬──────────────────┬────────────────────┤
|
||||
│ Web UI │ REST API │ CLI │
|
||||
│ (index.html) │ (api.php) │ (enterprise_pdf_ │
|
||||
│ Vanilla JS │ PHP endpoints │ checker.py) │
|
||||
│ Drag-drop │ Bearer/Key auth │ Direct Python │
|
||||
└─────────────────────┴──────────────────┴────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────┐
|
||||
│ api.php │
|
||||
│ (Router) │
|
||||
└──────┬───────┘
|
||||
│
|
||||
┌──────────────────┼──────────────────┐
|
||||
▼ ▼ ▼
|
||||
┌────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ File │ │ Redis Queue │ │ PostgreSQL │
|
||||
│ Upload │ │ (pdf:queue) │ │ (jobs, audit)│
|
||||
│uploads/│ │ │ │ │
|
||||
└────────┘ └──────┬───────┘ └──────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────┐
|
||||
│ worker.py │
|
||||
│ (Daemon process) │
|
||||
└──────┬───────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────┐
|
||||
│ EnterprisePDFChecker │
|
||||
│ • 30+ WCAG checks │
|
||||
│ • AI image analysis (Claude) │
|
||||
│ • OCR (GCV) │
|
||||
│ • Contrast analysis │
|
||||
│ • Readability metrics │
|
||||
└──────┬───────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────┐
|
||||
│ results/ │
|
||||
│ {job_id}. │
|
||||
│ result.json │
|
||||
└──────────────────┘
|
||||
```
|
||||
|
||||
## Dev Commands
|
||||
```bash
|
||||
# Tests
|
||||
source venv/bin/activate
|
||||
pytest tests/ -v # 31 tests
|
||||
pytest tests/ --cov=. --cov-report=html
|
||||
**Request Flow (Production Docker Stack):**
|
||||
1. User uploads PDF via web UI, REST API, or CLI
|
||||
2. `api.php` validates auth (Bearer token or API key via `auth.php`), saves file to `uploads/`
|
||||
3. Job created in PostgreSQL, queued to Redis (`pdf:queue`)
|
||||
4. `worker.py` daemon (background process) pops job, invokes `EnterprisePDFChecker.check_all()`
|
||||
5. All external API calls (Claude, GCV) wrapped with exponential backoff retry logic (`retry_helper.py`)
|
||||
6. Results written to `results/{job_id}.result.json`, PostgreSQL updated with completion status
|
||||
7. Client polls `api.php?action=status` for progress, fetches final results when ready
|
||||
8. Automatic cleanup (`cleanup.py`) removes uploads after 24h, results after 30 days
|
||||
|
||||
# Run locally
|
||||
php -S localhost:8000 # PHP dev server
|
||||
|
||||
# Docker
|
||||
docker-compose up # Dev
|
||||
docker-compose -f docker-compose.prod.yml up -d # Prod
|
||||
docker-compose exec worker pytest tests/ -v # Tests in container
|
||||
|
||||
# CLI usage
|
||||
python enterprise_pdf_checker.py doc.pdf --output report.json
|
||||
python enterprise_pdf_checker.py doc.pdf --quick # Skip AI checks
|
||||
python pdf_remediation.py doc.pdf --output fixed.pdf --all
|
||||
```
|
||||
|
||||
## Key Files
|
||||
**Key Source Files:**
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `enterprise_pdf_checker.py` | Core engine — 30+ WCAG checks |
|
||||
| `api.php` | REST API — file handling, job queue |
|
||||
| `auth.php` | Authentication — Bearer/X-API-Key |
|
||||
| `worker.py` | Background daemon — Redis queue consumer |
|
||||
| `pdf_remediation.py` | Auto-fix accessibility issues |
|
||||
| `enterprise_pdf_checker.py` | Core validation engine — 30+ WCAG checks, AI image analysis, scoring logic |
|
||||
| `api.php` | REST API router — upload/check/status/result/remediate/download endpoints, CORS headers |
|
||||
| `auth.php` | Authentication middleware — Bearer token, X-API-Key, dev mode localhost bypass |
|
||||
| `worker.py` | Background daemon — Redis queue consumer, graceful shutdown on signals |
|
||||
| `db_manager.py` | PostgreSQL ORM — jobs CRUD, audit logging, connection pooling |
|
||||
| `redis_queue.py` | Redis operations — job enqueue/dequeue, status tracking, rate limiting |
|
||||
| `pdf_remediation.py` | Auto-remediation — metadata fixing, language tagging, structural repairs |
|
||||
| `retry_helper.py` | Exponential backoff — retries for Claude API, GCV API, transient failures |
|
||||
| `report_generator.py` | Result formatting — JSON reports, HTML export, compliance summaries |
|
||||
| `logger_config.py` | Structured logging — JSON output, file rotation (10MB max), syslog integration |
|
||||
| `cleanup.py` | Scheduled task — file retention enforcement (24h uploads, 30d results) |
|
||||
| `index.html` | Web UI root — loads CSS/JS, sets up drag-drop zone, result viewer |
|
||||
| `js/app.js` | Frontend logic — API calls, progress polling, DOM updates, dark mode |
|
||||
| `css/style.css` | Branding — Oliver palette (black, #FFC407), Montserrat font, responsive layout |
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
# Activate virtual environment
|
||||
source venv/bin/activate
|
||||
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Run all tests (31 automated tests)
|
||||
pytest tests/ -v
|
||||
|
||||
# Run with coverage report
|
||||
pytest tests/ --cov=. --cov-report=html
|
||||
|
||||
# Run single test file
|
||||
pytest tests/test_checker.py -v
|
||||
|
||||
# Skip integration tests (faster local runs)
|
||||
pytest tests/ -m "not integration"
|
||||
|
||||
# Start development server (PHP)
|
||||
php -S localhost:8000
|
||||
|
||||
# CLI: Full accessibility check
|
||||
python enterprise_pdf_checker.py document.pdf --output report.json
|
||||
|
||||
# CLI: Quick check (skip AI image analysis)
|
||||
python enterprise_pdf_checker.py document.pdf --quick
|
||||
|
||||
# CLI: Auto-remediate all fixable issues
|
||||
python pdf_remediation.py document.pdf --output fixed.pdf --all
|
||||
|
||||
# Docker development stack (all services)
|
||||
docker-compose up
|
||||
|
||||
# Docker production stack
|
||||
docker-compose -f docker-compose.prod.yml up -d
|
||||
|
||||
# Run tests inside Docker container
|
||||
docker-compose exec worker pytest tests/ -v
|
||||
|
||||
# View worker logs
|
||||
docker-compose logs -f worker
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Server:** optical-web-1
|
||||
- **Deploy:**
|
||||
- **Development:** `docker-compose up`
|
||||
- **Production:** `docker-compose -f docker-compose.prod.yml up -d` OR via `deploy.sh` (runs git pull, restarts Apache)
|
||||
- **Manual:** Push to `git@bitbucket.org:zlalani/pdf-accessibility.git` main branch; server auto-deploys via webhook
|
||||
- **URL:** https://ai-sandbox.oliver.solutions/pdf-accessibility
|
||||
- **Port:** 8000 (development), 80/443 (production via Apache reverse proxy)
|
||||
- **Service:** None (Docker Compose manages container lifecycle; Apache may use systemd)
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/pdf-accessibility
|
||||
|
||||
## Environment Variables
|
||||
All configured in `.env` (copy from `.env.example`):
|
||||
|
||||
- `ANTHROPIC_API_KEY` — Anthropic Claude API key (required; get from https://console.anthropic.com/)
|
||||
- `GOOGLE_API_KEY` — Google Cloud Vision API key (optional; for OCR and text-in-images detection)
|
||||
- `GOOGLE_APPLICATION_CREDENTIALS` — Path to GCP service account JSON (alternative to API key)
|
||||
- `DEV_MODE` — Set to `true` for localhost auth bypass (development only)
|
||||
- `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASSWORD` — PostgreSQL connection (docker-compose provides defaults)
|
||||
- `CLOUD_RUN_URL` — Optional Cloud Run service URL for distributed PDF processing; defaults to local Python
|
||||
- `GCP_SA_KEY_PATH` — Path to GCP service account key for Cloud Run authentication
|
||||
- `GCS_BUCKET_NAME` — Google Cloud Storage bucket for page images (default: `optical-pdf-images`)
|
||||
- `RETENTION_HOURS` — Keep uploaded PDFs for N hours before deletion (default: 24)
|
||||
- `RESULTS_RETENTION_HOURS` — Keep result/meta JSON for N hours before deletion (default: 720 = 30 days)
|
||||
- `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_
|
||||
|
||||
## Timeline / Git History
|
||||
| Date | Change |
|
||||
|
|
@ -98,4 +201,4 @@ python pdf_remediation.py doc.pdf --output fixed.pdf --all
|
|||
|
||||
## Related
|
||||
- [[video-accessibility/Video Accessibility Platform]]
|
||||
- [[solventum-image-metadata/Solventum Image Metadata]]
|
||||
- [[solventum-image-metadata/Solventum Image Metadata]]
|
||||
|
|
@ -1,30 +1,196 @@
|
|||
---
|
||||
name: "Oliver DeckForge"
|
||||
client: Oliver Internal
|
||||
client: Oliver
|
||||
status: active
|
||||
server: optical-dev
|
||||
tech: [Next.js, FastAPI, Python, nginx, Docker]
|
||||
tech: [Next.js 14, FastAPI, PostgreSQL, Redis, Docker, React, Python, Puppeteer]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/ppt-tool
|
||||
deploy: docker compose up --build
|
||||
url:
|
||||
deploy: make dev
|
||||
url: https://optical-dev.oliver.solutions/ppt-tool/
|
||||
tags: [oliver, presentation, ai, nextjs, multi-tenant, rbac]
|
||||
created: 2026-04-14
|
||||
port: 80
|
||||
db: PostgreSQL
|
||||
---
|
||||
|
||||
## Overview
|
||||
AI-powered enterprise presentation generator. Multi-tenant architecture, custom templates, RBAC.
|
||||
|
||||
Architecture: `nginx :80` → `Next.js :3000` + `FastAPI :8000`
|
||||
**ppt-tool** (Oliver DeckForge) is an AI-powered enterprise presentation generator with multi-tenant architecture supporting custom template upload, role-based access control, and SSO authentication. Users upload documents, URLs, or topics and receive fully formatted slide decks in PPTX/PDF format. The system uses Claude/Gemini for content generation, Puppeteer for exports, and provides admin panels for user/client/team management with audit logging and analytics.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** Next.js (:3000)
|
||||
- **Backend:** FastAPI (:8000)
|
||||
- **Proxy:** nginx (:80)
|
||||
- **Infrastructure:** Docker + docker-compose
|
||||
|
||||
- **Frontend:** Next.js 14 (App Router), TypeScript, Redux Toolkit, Shadcn UI, react-i18next
|
||||
- **Backend:** FastAPI, SQLModel (SQLAlchemy ORM), Python 3.10+, Alembic (migrations)
|
||||
- **Database:** PostgreSQL 16 with asyncpg driver
|
||||
- **Infrastructure:** Docker Compose (6 services), nginx reverse proxy, shared app_data volume
|
||||
- **Job Queue:** arq (async Redis queue) for background AI generation tasks
|
||||
- **AI/ML:** Google Gemini 3.1 Pro (presentations), Gemini 2.5 Flash Lite (parsing), Anthropic Claude (fallback), OpenAI (fallback), Ollama (local)
|
||||
- **Image Generation:** Gemini Flash (primary), Pexels/Pixabay APIs (fallback)
|
||||
- **Export:** Puppeteer (headless Chromium) for PDF/PPTX generation
|
||||
- **Key libraries:** pptx (python-pptx), PyMuPDF (fitz), sqlalchemy, fastapi, alembic, arq, msal.js (Azure AD)
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────┐
|
||||
│ nginx │ :80
|
||||
└────┬────┘
|
||||
┌─────┴─────────────┐
|
||||
┌────┴───┐ ┌────┴────┐
|
||||
│ Next.js │ │ FastAPI │
|
||||
│ (web) │ │ (api) │
|
||||
│ :3000 │ │ :8000 │
|
||||
└────┬────┘ └───┬─────┘
|
||||
│ │
|
||||
│ ┌──────┴──────┐
|
||||
│ │ │
|
||||
│ ┌───┴───┐ ┌────┴─────┐
|
||||
│ │Worker │ │Postgres │
|
||||
│ │(arq) │ │ :5432 │
|
||||
│ └───┬───┘ └──────────┘
|
||||
│ │
|
||||
│ ┌───┴──────┐
|
||||
│ │ Redis │
|
||||
│ │ :6379 │
|
||||
│ └──────────┘
|
||||
│
|
||||
└─────────────────────── app_data volume
|
||||
(shared storage)
|
||||
```
|
||||
|
||||
**Service breakdown:**
|
||||
- **web (Next.js):** SPA frontend with Puppeteer-based local PDF/PPTX export, Redux state, SSE for real-time updates
|
||||
- **api (FastAPI):** REST API (v1), auth/RBAC, SSE streaming endpoints, template parsing, client/user management
|
||||
- **worker (arq):** Consumes Redis jobs from API, runs AI generation tasks in background, writes results to app_data
|
||||
- **postgres:** Primary data store for users, clients, presentations, slides, templates, audit logs
|
||||
- **redis:** Job queue for arq worker tasks, caching layer
|
||||
- **nginx:** Routes /api/v1/* → FastAPI, / → Next.js, serves static app_data files
|
||||
|
||||
**Data flow:** User creates presentation → API enqueues arq job → Worker generates slides via Claude/Gemini → Results stored in app_data + DB → Frontend polls SSE or fetches results → User exports via Puppeteer (web) or downloads from API.
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
# Start all services (build + up)
|
||||
make dev
|
||||
|
||||
# Build images only (no start)
|
||||
make build
|
||||
|
||||
# Start containers in background
|
||||
make up
|
||||
|
||||
# Stop all containers
|
||||
make down
|
||||
|
||||
# Run database migrations
|
||||
make migrate
|
||||
|
||||
# Seed database with sample data
|
||||
make seed
|
||||
|
||||
# Run API tests
|
||||
make test
|
||||
|
||||
# Run E2E tests (Cypress)
|
||||
make test-e2e
|
||||
|
||||
# Run all tests
|
||||
make test-all
|
||||
|
||||
# View logs from all services
|
||||
make logs
|
||||
|
||||
# Open bash shell in API container
|
||||
make shell-api
|
||||
|
||||
# Open psql shell in Postgres container
|
||||
make shell-db
|
||||
```
|
||||
|
||||
**Local access:**
|
||||
- App: http://localhost (nginx) or http://localhost:3000 (direct)
|
||||
- API: http://localhost:8000
|
||||
- Postgres: localhost:5432
|
||||
- Redis: localhost:6379
|
||||
|
||||
## Deployment
|
||||
- **Run:** `docker compose up --build`
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/ppt-tool`
|
||||
|
||||
- **Server:** optical-dev (optical-dev.oliver.solutions)
|
||||
- **Deploy:** `make dev` (local) or custom Docker stack (production uses subpath `/ppt-tool/`)
|
||||
- **URL:** https://optical-dev.oliver.solutions/ppt-tool/
|
||||
- **Port:** 80 (nginx) / 8000 (API internal) / 3000 (Next.js internal)
|
||||
- **Service:** None (Docker Compose manages services)
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/ppt-tool
|
||||
|
||||
**Deployment notes:**
|
||||
- App runs under subpath `/ppt-tool/` on production (basePath hardcoded in Next.js config)
|
||||
- Docker build args pass Azure AD credentials: `AZURE_AD_TENANT_ID`, `AZURE_AD_CLIENT_ID`
|
||||
- Shared `app_data` volume persists uploads, exports, and generated files across containers
|
||||
- Alembic migrations auto-run on API startup (or `make migrate`)
|
||||
|
||||
## Environment Variables
|
||||
|
||||
**Auth & Security:**
|
||||
- `AZURE_AD_TENANT_ID` — Azure AD tenant ID (leave blank for dev bypass mode)
|
||||
- `AZURE_AD_CLIENT_ID` — Azure AD app registration client ID
|
||||
- `AZURE_AD_CLIENT_SECRET` — Azure AD app registration secret
|
||||
- `AZURE_AD_REDIRECT_URI` — OAuth2 callback URL (e.g., https://yourdomain.com/api/v1/auth/callback)
|
||||
- `JWT_SECRET_KEY` — Secret for signing JWT tokens (must be 256-bit, change in production)
|
||||
- `DEV_AUTH_PASSWORD` — Password for dev auth bypass (only used when Azure AD is disabled)
|
||||
- `ALLOWED_ORIGINS` — CORS whitelist (comma-separated; local: http://localhost:3000,http://localhost)
|
||||
|
||||
**AI & Models:**
|
||||
- `GOOGLE_API_KEY` — Google AI (Gemini) API key from https://aistudio.google.com/app/apikey
|
||||
- `GOOGLE_MODEL` — Primary model for presentation generation (default: gemini-3.1-pro-preview)
|
||||
- `PARSING_MODEL` — Fast model for master deck parsing (default: gemini-2.5-flash-lite)
|
||||
- `IMAGE_PROVIDER` — Image generation provider (default: gemini_flash)
|
||||
- `PEXELS_API_KEY` — Fallback image source (optional)
|
||||
- `PIXABAY_API_KEY` — Fallback image source (optional)
|
||||
- `DISABLE_IMAGE_GENERATION` — Set to "true" to disable all image generation
|
||||
|
||||
**Database & Storage:**
|
||||
- `POSTGRES_PASSWORD` — Postgres user password (default: deckforge)
|
||||
- `DATABASE_URL` — Full connection string (auto-set in docker-compose)
|
||||
- `REDIS_URL` — Redis connection string (default: redis://redis:6379/0)
|
||||
- `APP_DATA_DIRECTORY` — Path for uploads/exports (default: /app_data)
|
||||
- `TEMP_DIRECTORY` — Temp file path (default: /tmp/deckforge)
|
||||
|
||||
**Connection Pooling:**
|
||||
- `DB_POOL_SIZE` — SQLAlchemy pool size (default: 20)
|
||||
- `DB_MAX_OVERFLOW` — Max overflow connections (default: 40)
|
||||
- `DB_POOL_RECYCLE` — Recycle connections after N seconds (default: 3600)
|
||||
|
||||
**App Settings:**
|
||||
- `CAN_CHANGE_KEYS` — Allow runtime API key updates (default: false)
|
||||
- `DISABLE_ANONYMOUS_TRACKING` — Disable telemetry (default: true)
|
||||
- `MAX_REQUEST_SIZE` — Max upload size in bytes (default: 104857600 = 100MB)
|
||||
|
||||
## API / Endpoints
|
||||
|
||||
**Core endpoints** (base: http://api:8000/api/v1):
|
||||
- `POST /auth/callback` — Azure AD OAuth2 callback
|
||||
- `POST /presentations` — Create new presentation
|
||||
- `GET /presentations` — List user presentations
|
||||
- `GET /presentations/{id}` — Get presentation details
|
||||
- `POST /presentations/{id}/export` — Export to PDF/PPTX
|
||||
- `GET /presentations/{id}/events` — SSE stream for real-time generation updates
|
||||
- `POST /templates` — Upload master PPTX deck
|
||||
- `GET /templates` — List templates
|
||||
- `POST /clients` — Create client (admin only)
|
||||
- `GET /users` — List users (admin/client admin)
|
||||
- `POST /audit-logs` — Fetch audit logs (admin only)
|
||||
|
||||
**Frontend internal communication:**
|
||||
- API calls use `apiFetch()` helper (prefixes `/ppt-tool/api/v1/` basePath for subpath deployment)
|
||||
- SSE via EventSource with URL: `/ppt-tool/api/v1/presentations/{id}/events`
|
||||
|
||||
## Known Issues
|
||||
|
||||
- **Azure AD:** Dev bypass mode requires manual password; change `DEV_AUTH_PASSWORD` in `.env` or enable proper Azure AD config
|
||||
- **Image generation:** Gemini image generation occasionally fails; fallback to Pexels/Pixabay if needed (set API keys)
|
||||
- **PyMuPDF text extraction:** Some complex PDFs may not extract cleanly; fallback to DOCX upload recommended
|
||||
- **Large
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
|
|
@ -37,4 +203,4 @@ Architecture: `nginx :80` → `Next.js :3000` + `FastAPI :8000`
|
|||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
|
||||
## Related
|
||||
- [[presenton/Presenton]]
|
||||
- [[presenton/Presenton]]
|
||||
|
|
@ -2,24 +2,186 @@
|
|||
name: "Presenton"
|
||||
client: Oliver Internal
|
||||
status: active
|
||||
tech: [Node.js, Docker]
|
||||
tech: [Node.js, Python, Docker, OpenAI API, Gemini, React, Tailwind CSS]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/presenton
|
||||
deploy: docker compose up --build
|
||||
deploy: docker-compose up
|
||||
url:
|
||||
tags: [oliver, presentation, ai, open-source]
|
||||
created: 2026-04-14
|
||||
server: local
|
||||
port: 5000
|
||||
db: PostgreSQL
|
||||
---
|
||||
|
||||
## Overview
|
||||
Open-source AI Presentation Generator and API — alternative to Gamma, Beautiful.AI, Decktopus. Community: Discord + Twitter/X.
|
||||
Presenton is an open-source AI presentation generator that creates professional presentations locally on your device using AI models. It serves as an alternative to Gamma, Beautiful AI, and Decktopus, offering users complete control over their data and privacy. The platform supports multiple LLM providers (OpenAI, Google Gemini, Anthropic Claude, Ollama) and image generation sources (DALL-E 3, Gemini Flash, Pexels, Pixabay, ComfyUI), allowing users to generate presentations from prompts, documents, or existing PPTX templates. Key capabilities include custom HTML/Tailwind CSS templates, PPTX/PDF export, MCP server integration, and flexible deployment options.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** Node.js
|
||||
- **Infrastructure:** Docker + docker-compose
|
||||
- **Frontend:** React, Tailwind CSS, HTML5
|
||||
- **Backend:** Node.js, Python
|
||||
- **Database:** PostgreSQL
|
||||
- **Infrastructure:** Docker, Docker Compose (with optional GPU support via NVIDIA)
|
||||
- **AI/ML:** OpenAI API, Google Gemini, Anthropic Claude, Ollama (local models), custom OpenAI-compatible endpoints, ComfyUI
|
||||
- **Key libraries:** Model Context Protocol (MCP), various image generation APIs (DALL-E 3, Gemini Flash, Pexels, Pixabay, NanoBanana)
|
||||
|
||||
## Architecture
|
||||
Presenton follows a containerized microservices approach with Docker Compose orchestration:
|
||||
|
||||
**Core Components:**
|
||||
1. **Frontend UI** — React-based web interface for users to input prompts, upload documents/PPTX files, and configure settings
|
||||
2. **Backend API** — Node.js/Python API service that orchestrates presentation generation
|
||||
3. **LLM Integration Layer** — Abstracts multiple LLM providers (OpenAI, Gemini, Claude, Ollama, custom endpoints)
|
||||
4. **Template Engine** — Loads and processes HTML/Tailwind CSS templates; can generate templates from existing PPTX files
|
||||
5. **Image Generation Layer** — Multi-provider image generation (DALL-E 3, Gemini Flash, Pexels, Pixabay, ComfyUI, NanoBanana)
|
||||
6. **Export Engine** — Converts generated presentations to PPTX and PDF formats
|
||||
7. **MCP Server** — Built-in Model Context Protocol server for generating presentations programmatically
|
||||
8. **Database Layer** — PostgreSQL for persistence (user settings, templates, generation history)
|
||||
|
||||
**Data Flow:**
|
||||
User Input (prompt/document/PPTX) → Backend API → LLM Provider → Content Generation → Image Generation → Template Rendering → PPTX/PDF Export → User Download
|
||||
|
||||
**Deployment Options:**
|
||||
- Development: `docker-compose` with Dockerfile.dev (hot reload)
|
||||
- Production: `docker-compose` with Dockerfile (optimized image)
|
||||
- GPU-enabled: Production service with NVIDIA GPU reservation
|
||||
- API Deployment: Can be hosted as a standalone service for team use
|
||||
|
||||
```
|
||||
┌─────────────┐
|
||||
│ Frontend │ (React/Tailwind)
|
||||
└──────┬──────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────┐
|
||||
│ Backend API (Node.js/Python) │
|
||||
│ ┌──────────────────────────┐ │
|
||||
│ │ LLM Abstraction Layer │ │
|
||||
│ │ (OpenAI/Gemini/Claude) │ │
|
||||
│ └──────────────────────────┘ │
|
||||
│ ┌──────────────────────────┐ │
|
||||
│ │ Template Engine + Parser │ │
|
||||
│ └──────────────────────────┘ │
|
||||
│ ┌──────────────────────────┐ │
|
||||
│ │ Image Generation Layer │ │
|
||||
│ │ (DALL-E/Gemini/etc) │ │
|
||||
│ └──────────────────────────┘ │
|
||||
└──────┬───────────────────────────┘
|
||||
│
|
||||
├─► PostgreSQL (persistence)
|
||||
│
|
||||
└─► PPTX/PDF Export
|
||||
```
|
||||
|
||||
## Dev Commands
|
||||
```bash
|
||||
# Start development server (with hot reload)
|
||||
docker-compose up development
|
||||
|
||||
# Start production server (CPU)
|
||||
docker-compose up production
|
||||
|
||||
# Start production server (with GPU support)
|
||||
docker-compose up production-gpu
|
||||
|
||||
# Build Docker image
|
||||
docker build -t presenton:latest .
|
||||
|
||||
# Build development image
|
||||
docker build -f Dockerfile.dev -t presenton:dev .
|
||||
|
||||
# View logs
|
||||
docker-compose logs -f
|
||||
|
||||
# Access web interface
|
||||
# Navigate to http://localhost:5000
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `docker compose up --build`
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/presenton`
|
||||
- **Server:** local
|
||||
- **Deploy:** `docker-compose up production` (or `production-gpu` for GPU support)
|
||||
- **URL:** http://localhost:5000 (default)
|
||||
- **Port:** 5000 (configurable in docker-compose.yml)
|
||||
- **Service:** Docker Compose managed service
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/presenton
|
||||
- **Volume mounts:** `./app_data:/app_data` (persistent data storage)
|
||||
|
||||
**Deployment Notes:**
|
||||
- Change port in `docker-compose.yml` by modifying the port binding (e.g., `"8080:80"`)
|
||||
- GPU deployment requires NVIDIA Docker runtime and `nvidia-docker`
|
||||
- Persistent data stored in `./app_data` directory
|
||||
- Can be deployed on any Docker-capable infrastructure (cloud, on-premise, local)
|
||||
|
||||
## Environment Variables
|
||||
Core LLM Configuration:
|
||||
- `LLM` — Primary LLM provider (`openai`, `gemini`, `claude`, `ollama`, `custom`)
|
||||
- `OPENAI_API_KEY` — OpenAI API key
|
||||
- `OPENAI_MODEL` — OpenAI model name (e.g., `gpt-4-turbo`)
|
||||
- `GOOGLE_API_KEY` — Google API key for Gemini
|
||||
- `GOOGLE_MODEL` — Gemini model name
|
||||
- `ANTHROPIC_API_KEY` — Anthropic Claude API key
|
||||
- `ANTHROPIC_MODEL` — Claude model name
|
||||
- `OLLAMA_URL` — Ollama server URL for local models
|
||||
- `OLLAMA_MODEL` — Ollama model name
|
||||
|
||||
Custom LLM:
|
||||
- `CUSTOM_LLM_URL` — Custom OpenAI-compatible endpoint URL
|
||||
- `CUSTOM_LLM_API_KEY` — API key for custom endpoint
|
||||
- `CUSTOM_MODEL` — Model name on custom endpoint
|
||||
|
||||
Image Generation:
|
||||
- `PEXELS_API_KEY` — Pexels stock image API key
|
||||
- `COMFYUI_URL` — ComfyUI server URL for local image generation
|
||||
- `COMFYUI_WORKFLOW` — ComfyUI workflow configuration
|
||||
|
||||
Advanced Features:
|
||||
- `EXTENDED_REASONING` — Enable extended reasoning in LLM
|
||||
- `TOOL_CALLS` — Enable function/tool calling in LLM
|
||||
- `DISABLE_THINKING` — Disable model thinking/reasoning
|
||||
- `WEB_GROUNDING` — Enable web search grounding for content generation
|
||||
- `DISABLE_ANONYMOUS_TRACKING` — Disable anonymous usage telemetry
|
||||
|
||||
Infrastructure:
|
||||
- `DATABASE_URL` — PostgreSQL connection string
|
||||
- `CAN_CHANGE_KEYS` — Allow users to change API keys in UI (true/false)
|
||||
|
||||
## API / Endpoints
|
||||
Presenton exposes a REST API for presentation generation and includes a built-in MCP (Model Context Protocol) server.
|
||||
|
||||
**Key API Endpoints** (inferred from codebase):
|
||||
- `POST /api/presentations/generate` — Generate presentation from prompt or document
|
||||
- `POST /api/presentations/template` — Generate template from uploaded PPTX
|
||||
- `POST /api/export` — Export presentation to PPTX/PDF
|
||||
- `GET /api/templates` — List available templates
|
||||
- `POST /api/settings` — Update user API key settings
|
||||
- `GET /health` — Health check endpoint
|
||||
|
||||
**MCP Server Integration:**
|
||||
- Built-in MCP server for programmatic presentation generation
|
||||
- Allows integration with Claude and other MCP-compatible tools
|
||||
|
||||
See `/docs` (https://docs.presenton.ai) for complete API documentation.
|
||||
|
||||
## Known Issues
|
||||
- No explicit issues documented in provided files
|
||||
- Icon fallback mechanism in place for missing slide icons (commit d623fa6)
|
||||
- Recent API parameter rename from 'prompt' to 'content' (commit c5e36ab)
|
||||
- Some MCP/OpenAI spec compatibility fixes implemented (commit 932c807)
|
||||
|
||||
**Notable Recent Changes:**
|
||||
- New template system and refactored template loading (v1.0.0+)
|
||||
- Added GPT Image 1.5 support
|
||||
- NanoBanana image generation integration
|
||||
- ComfyUI local image generation workflow support
|
||||
- Ukrainian language support added
|
||||
- Multiple image provider options (DALL-E 3, Gemini Flash, Pexels, Pixabay)
|
||||
|
||||
## Git
|
||||
- **Remote:** https://github.com/presenton/presenton.git
|
||||
- **Latest commit:** acb850b (feat: New Templates and refactor template loading)
|
||||
- **Branch:** presenton/feat/new_templates_and_refactor_template_loading
|
||||
- **Community:** Discord (https://discord.gg/9ZsKKxudNE), X/Twitter (@presentonai)
|
||||
- **License:** Apache 2.0 (open-source)
|
||||
- **Contact:** suraj@presenton.ai (enterprise inquiries)
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
|
|
@ -32,4 +194,4 @@ Open-source AI Presentation Generator and API — alternative to Gamma, Beautifu
|
|||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
|
||||
## Related
|
||||
- [[ppt-tool/PPT Tool DeckForge]]
|
||||
- [[ppt-tool/PPT Tool DeckForge]]
|
||||
|
|
@ -1,27 +1,133 @@
|
|||
---
|
||||
name: "SmartCrop26"
|
||||
client: Oliver Internal
|
||||
client: Oliver Solutions
|
||||
status: active
|
||||
server: optical-web-1
|
||||
tech: [React, TypeScript, Vite, Lovable]
|
||||
server: ubuntu-server
|
||||
tech: [React 18, TypeScript, Vite, Tailwind CSS, shadcn-ui, Google Gemini API, Azure AD]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/smartcrop26
|
||||
deploy: npm run dev
|
||||
url:
|
||||
deploy: /opt/smartcrop26/deploy.sh
|
||||
url: https://ai-sandbox.oliver.solutions/smartcrop26
|
||||
tags: [oliver, smartcrop, image, lovable, react]
|
||||
created: 2026-04-14
|
||||
port: 8080
|
||||
---
|
||||
|
||||
## Overview
|
||||
Smart image cropping tool built via Lovable.dev. Changes committed automatically from Lovable visual editor.
|
||||
|
||||
**SmartCrop2026** is a fully client-side, AI-powered image cropping tool built with React 18 and TypeScript. Users upload images and select aspect ratios; the app suggests optimal crop regions using either Google Gemini 2.5 Flash API or local browser-based Sobel edge detection. Crops are previewed interactively, edited manually, and exported as resized PNGs bundled into a ZIP file. It requires Azure AD SSO login and serves Oliver Solutions' internal workflows.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** React + TypeScript + Vite
|
||||
- **Platform:** Lovable.dev
|
||||
|
||||
- **Frontend:** React 18, TypeScript, Vite, Tailwind CSS 3.4, shadcn-ui (Radix primitives), Lucide icons
|
||||
- **Backend:** None (fully client-side)
|
||||
- **Database:** None
|
||||
- **Infrastructure:** Ubuntu server deployment via bash script; served at `https://ai-sandbox.oliver.solutions/smartcrop26`
|
||||
- **AI/ML:** Google Gemini 2.5 Flash (API-based analysis) + Sobel edge detection (browser-based fallback)
|
||||
- **Key libraries:** JSZip (ZIP export), FileSaver (download), MSAL (Azure AD auth), React Hook Form (forms), React Query (lightly configured)
|
||||
|
||||
## Architecture
|
||||
|
||||
**Central orchestrator:** `src/pages/Index.tsx` manages all top-level state:
|
||||
- `images[]` — uploaded image data URLs
|
||||
- `selectedRatios[]` — chosen aspect ratios
|
||||
- `cropsMap{}` — crop suggestions per image
|
||||
- `engine` toggle — AI vs. Local analysis mode
|
||||
|
||||
**Dual analysis engines:**
|
||||
- **AI engine** (`lib/analyze-image.ts`): Encodes image as base64, sends to Google Gemini via function calling, returns `CropSuggestion[]` with normalized coords (0–1)
|
||||
- **Local engine** (`lib/analyze-local.ts`): Sobel edge detection in-browser; scales image to max 400px, samples candidate windows with center-weight bias
|
||||
|
||||
**UI components:**
|
||||
- `ImageUpload` — FileReader for image ingestion
|
||||
- `RatioSelector` — 12 built-in platform-grouped ratios + custom input
|
||||
- `CropPreviewCard` — Canvas-rendered thumbnails
|
||||
- `CropEditor` — Interactive crop region dragging/resizing via Canvas
|
||||
|
||||
**Export pipeline** (`lib/export-zip.ts`):
|
||||
1. Iterate all uploaded images + selected crops
|
||||
2. Rescale each crop to requested pixel sizes via Canvas
|
||||
3. Pack PNGs into JSZip archive
|
||||
4. Trigger browser download via FileSaver
|
||||
|
||||
**Config parsing** (`lib/crop-types.ts`):
|
||||
- Handles current v2 JSON, legacy Python tool format, and simple ratio lists
|
||||
- `PRESET_RATIOS` defines 12 platform groups (Instagram, TikTok, LinkedIn, etc.)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Index.tsx (State) │
|
||||
│ images[] | selectedRatios[] | cropsMap{} | engine │
|
||||
└────┬──────────────────────────────────────────────┬─────┘
|
||||
│ │
|
||||
┌────▼──────────┐ ┌──────────────────┐ ┌────────▼──┐
|
||||
│ ImageUpload │ │ RatioSelector │ │ CropEditor│
|
||||
│ (FileReader) │ │ (PRESET_RATIOS) │ │ (Canvas) │
|
||||
└───────────────┘ └──────────────────┘ └───────────┘
|
||||
│ │
|
||||
┌────▼────────────────────────────────────────────▼─────┐
|
||||
│ Dual Analysis Engines │
|
||||
│ ┌──────────────────┐ ┌──────────────────────┐ │
|
||||
│ │ AI (Gemini) │ │ Local (Sobel) │ │
|
||||
│ │ base64 → API │ │ in-browser EdgeDet │ │
|
||||
│ └──────────────────┘ └──────────────────────┘ │
|
||||
│ ↓ CropSuggestion[] (normalized 0–1) │
|
||||
└──────────────────────────────────────────────────────┘
|
||||
│
|
||||
┌────▼──────────────┐
|
||||
│ CropPreviewCard │
|
||||
│ (Canvas render) │
|
||||
└─────────┬────────┘
|
||||
│
|
||||
┌────▼──────────────────┐
|
||||
│ export-zip.ts │
|
||||
│ (Canvas rescale + ZIP)│
|
||||
└─────────┬─────────────┘
|
||||
(download)
|
||||
```
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
npm run dev # Start dev server at http://localhost:8080
|
||||
npm run build # Production build (dist/)
|
||||
npm run build:dev # Dev build with source maps
|
||||
npm run lint # Run ESLint
|
||||
npm run test # Vitest single run
|
||||
npm run test:watch # Vitest watch mode
|
||||
npm run preview # Preview production build locally
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Dev:** `npm run dev`
|
||||
|
||||
- **Server:** Ubuntu (internal Oliver Solutions infrastructure)
|
||||
- **Deploy:** Run `/opt/smartcrop26/deploy.sh` as root or with sudo
|
||||
- Script pulls from git, runs `npm ci && npm run build`, copies `dist/` to `/var/www/html/smartcrop26/`
|
||||
- **URL:** `https://ai-sandbox.oliver.solutions/smartcrop26`
|
||||
- **Port:** 8080 (dev); served via web server on production
|
||||
- **Service:** None (static site; no systemd service)
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/smartcrop26`
|
||||
|
||||
## Environment Variables
|
||||
|
||||
- `VITE_GOOGLE_API_KEY` — Google Gemini 2.5 Flash API key; required for AI analysis engine
|
||||
- `VITE_AZURE_TENANT_ID` — Azure AD tenant ID (`e519c2e6-bc6d-4fdf-8d9c-923c2f002385`)
|
||||
- `VITE_AZURE_CLIENT_ID` — Azure AD app registration client ID (`9079054c-9620-4757-a256-23413042f1ef`)
|
||||
- `VITE_AZURE_REDIRECT_URI` — Azure AD redirect after login (`https://ai-sandbox.oliver.solutions/smartcrop26`)
|
||||
|
||||
See `.env.example` for template.
|
||||
|
||||
## Known Issues
|
||||
|
||||
- TypeScript config is loose (`noImplicitAny: false`, no strict null checks) — reduces type safety but allows faster iteration
|
||||
- TanStack React Query is installed but lightly used; consider full adoption for server state
|
||||
- Azure AD auth is required; no anonymous/guest mode
|
||||
- Gemini API calls require valid `VITE_GOOGLE_API_KEY` or falls back to local Sobel engine
|
||||
|
||||
## Git
|
||||
|
||||
- **Remote:** `git@bitbucket.org:zlalani/smartcrop26.git`
|
||||
- **Recent commits:** Add Reset/Remove buttons (24883a3), batch generate + ZIP fixes (66bafdc), login redesign (222ebe8), Azure SSO auth (f145712)
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
**Done:** Added to Obsidian second brain.
|
||||
|
|
@ -30,4 +136,4 @@ Smart image cropping tool built via Lovable.dev. Changes committed automatically
|
|||
## Change Log
|
||||
| Date | Requested | Changed | Files |
|
||||
|------|-----------|---------|-------|
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
|
|
@ -2,27 +2,145 @@
|
|||
name: "Social Reporting Tool"
|
||||
client: "TBD"
|
||||
status: active
|
||||
server: optical-dev
|
||||
tech: []
|
||||
server: local
|
||||
tech: [TypeScript, Node.js, React, PostgreSQL, Claude AI, Apify, Docker, Apache]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/social-reporting-tool
|
||||
deploy:
|
||||
url:
|
||||
deploy: docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
|
||||
url: https://your-domain.com/social-reports/
|
||||
tags:
|
||||
- project
|
||||
created: 2026-04-15
|
||||
port: 3456
|
||||
db: PostgreSQL
|
||||
---
|
||||
|
||||
## Overview
|
||||
> New project — fill in during first session.
|
||||
Social Reporting Tool is an automated social media research platform that scrapes TikTok, Instagram, and YouTube via Apify, analyzes content with Claude AI, and generates client-ready HTML reports. It serves marketing teams and research agencies who need rapid competitive intelligence and trend analysis across multiple social platforms. The system combines real-time scraping with AI-driven content analysis in an 8-stage pipeline, offering a web dashboard for brief management, progress tracking, and cost monitoring.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:**
|
||||
- **Backend:**
|
||||
- **Infrastructure:**
|
||||
- **Frontend:** HTML/CSS/JavaScript (vanilla), MSAL.js for Azure AD SSO
|
||||
- **Backend:** Node.js 20+, TypeScript, tsx (runtime)
|
||||
- **Database:** PostgreSQL 16 (Alpine)
|
||||
- **Infrastructure:** Docker Compose, Apache (production reverse proxy)
|
||||
- **AI/ML:** Anthropic Claude API (content analysis, trend detection, report generation)
|
||||
- **Key libraries:** postgres (Node.js driver), Apify SDK (web scraping), native Node.js HTTP
|
||||
|
||||
## Architecture
|
||||
The system is organized as a Node.js backend with a static frontend, powered by an 8-stage pipeline:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ Frontend (Static HTML/JS) │
|
||||
│ (Brief Entry, Dashboard, Runs, History, Help Tabs)│
|
||||
└─────────────┬───────────────────────────────────────┘
|
||||
│ HTTP + SSE
|
||||
┌─────────────▼───────────────────────────────────────┐
|
||||
│ Node.js Dashboard Server (port 3456) │
|
||||
│ • Session auth (HMAC cookie-based) │
|
||||
│ • Brief management (save/load/list) │
|
||||
│ • Pipeline orchestration & SSE progress │
|
||||
│ • Run history + cost tracking │
|
||||
└─────────────┬───────────────────────────────────────┘
|
||||
│ SQL
|
||||
┌─────────────▼───────────────────────────────────────┐
|
||||
│ PostgreSQL (social_listening) │
|
||||
│ • Runs table (status, costs, brief data) │
|
||||
│ • Brief storage (JSON documents) │
|
||||
└─────────────────────────────────────────────────────┘
|
||||
|
||||
Pipeline (run.ts):
|
||||
Stage 1: Brief Validation → normalize and validate client brief
|
||||
Stage 2: Strategy Review → Claude suggests 3 extra hashtags
|
||||
Stage 3: Discovery Scrape → Apify scrapes TikTok/Instagram/YouTube
|
||||
Stage 4: Data Review → Claude analyzes trends from scraped content
|
||||
Stage 5: Enrichment Scrape → fetch transcripts and metadata
|
||||
Stage 6: Pre-Report Review → Claude refines findings
|
||||
Stage 7: Desk Research → web search for context
|
||||
Stage 8: Report Generation → HTML report with video embeds
|
||||
```
|
||||
|
||||
**Key Design Decisions:**
|
||||
- Apify scraping runs **sequentially** (not parallel) to prevent budget overruns
|
||||
- Budget control via `APIFY_COST_LIMIT` — pipeline stops when limit is reached
|
||||
- Per-brief Apify budget with platform cost splitting
|
||||
- SSE real-time progress updates with live cost tracking
|
||||
- Run outputs stored as JSON; reports as HTML with native embeds (YouTube iframes, Instagram embeds, TikTok links)
|
||||
- Session-based auth with HMAC-signed tokens; optional Azure AD SSO
|
||||
|
||||
## Dev Commands
|
||||
```bash
|
||||
# Local setup
|
||||
npm install
|
||||
|
||||
# Start PostgreSQL + app via Docker
|
||||
docker compose up -d
|
||||
|
||||
# Dashboard runs at http://localhost:3456
|
||||
|
||||
# CLI commands (without dashboard)
|
||||
npm run pipeline # dry run (no actual scraping)
|
||||
npm run pipeline:test # test mode (mock data)
|
||||
npm run pipeline:live # live Apify scraping (requires APIFY_LIVE_APPROVED=true)
|
||||
|
||||
# Start just the dashboard server (if DB already running)
|
||||
npm run dashboard
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Server:** Ubuntu server (any; configured for Apache reverse proxy)
|
||||
- **Deploy:**
|
||||
```bash
|
||||
cd /opt/social-reporting
|
||||
git pull
|
||||
cp frontend/* /var/www/html/social-reporting/
|
||||
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
|
||||
```
|
||||
- **URL:** `https://your-domain.com/social-reports/`
|
||||
- **Port:** 3456 (backend, proxied via Apache)
|
||||
- **Service:** Docker Compose (not systemd)
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/social-reporting-tool`
|
||||
|
||||
## Environment Variables
|
||||
- `APIFY_TOKEN` — Apify API token (required for scraping)
|
||||
- `ANTHROPIC_API_KEY` — Claude API key (required)
|
||||
- `APIFY_LIVE_APPROVED` — `true` to enable real Apify scraping; `false` for dry run
|
||||
- `APIFY_COST_LIMIT` — USD budget limit per run (default: 5)
|
||||
- `TEST_MODE` — `true` to use mock data instead of real APIs
|
||||
- `DASHBOARD_PORT` — port for Node.js backend (default: 3456)
|
||||
- `DATABASE_URL` — PostgreSQL connection string
|
||||
- `DASH_USER` / `DASH_PASS` — basic auth credentials for dashboard
|
||||
- `SESSION_SECRET` — random secret for HMAC cookie signing
|
||||
- `ALLOWED_ORIGIN` — CORS origin (e.g., for production domain)
|
||||
- `AZURE_TENANT_ID` / `AZURE_CLIENT_ID` — optional Azure AD SSO config
|
||||
- `DB_PASSWORD` — PostgreSQL password (used in docker-compose)
|
||||
|
||||
## API / Endpoints
|
||||
All endpoints run on the dashboard server (default `:3456`):
|
||||
|
||||
- `POST /run` — start a new pipeline run (request body: brief JSON)
|
||||
- `GET /runs` — list all historical runs with costs and status
|
||||
- `GET /runs/:id` — fetch specific run details and report
|
||||
- `DELETE /runs/:id` — delete a run
|
||||
- `POST /briefs` — save a new brief (request body: brief JSON)
|
||||
- `GET /briefs` — list all saved briefs
|
||||
- `GET /briefs/:id` — load a specific brief
|
||||
- `DELETE /briefs/:id` — delete a brief
|
||||
- `GET /events` — SSE stream (real-time pipeline progress + costs)
|
||||
|
||||
Session auth via `session` cookie; responses include cost breakdowns per platform.
|
||||
|
||||
## Known Issues
|
||||
- **Unicode/Claude API**: Fixed in commit ce916cd (ensure UTF-8 encoding in all API calls)
|
||||
- **SSE reconnect**: Fixed in commit 087d1bb (POST /run only once per pipeline start)
|
||||
- **Apify budget**: Runs sequentially to prevent overruns; parallel scraping disabled
|
||||
- **Frontend deployment**: Apache must copy frontend files to `/var/www/html` on deploy (fixed in 7a70283)
|
||||
- **No documented TODOs or breaking issues** as of latest commit
|
||||
|
||||
## Git
|
||||
- **Remote:** `git@bitbucket.org:zlalani/social-reporting-tool.git`
|
||||
- **Default branch:** main (assumed)
|
||||
- **Recent focus:** Azure AD SSO, security hardening, report quality, brief management, cost tracking
|
||||
|
||||
## Sessions
|
||||
### 2026-04-15 – Fix frontend deployment to /var/www/html and
|
||||
**Asked:** Fix frontend deployment to /var/www/html and ensure Apache reloads after copy.
|
||||
|
|
@ -49,4 +167,4 @@ created: 2026-04-15
|
|||
| 2026-04-15 | SSO setup | Added MSAL.js library and Azure AD configuration, enabled SSO token exchange | frontend/msal-browser.min.js, frontend/config.js, frontend/login.html |
|
||||
| 2026-04-15 | SSO setup with Azure AD | Azure tenant ID, client ID, redirect URI configured; server.ts verified | server.ts |
|
||||
|
||||
## Related
|
||||
## Related
|
||||
|
|
@ -1,31 +1,183 @@
|
|||
---
|
||||
name: "Solventum Image Metadata Tool v3.1"
|
||||
client: Solventum
|
||||
client: Solventum / Oliver Marketing
|
||||
status: active
|
||||
server: optical-web-1
|
||||
tech: [Python, FastAPI, Docker, OpenAI, HTML/JS]
|
||||
server: ai-sandbox.oliver.solutions
|
||||
tech: [Python, FastAPI, Flask, SQLite, ExifTool, OpenAI, Azure AD]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/solventum-image-metadata
|
||||
deploy: docker compose up --build
|
||||
url:
|
||||
deploy: docker-compose up -d
|
||||
url: https://ai-sandbox.oliver.solutions/solventum-image-metadata
|
||||
tags: [solventum, metadata, ai, openai, enterprise]
|
||||
created: 2026-04-14
|
||||
port: 5001
|
||||
db: SQLite
|
||||
---
|
||||
|
||||
## Overview
|
||||
Universal metadata creation and management tool (Enterprise Edition v3.1). Supports CSV/Excel/JSON import, AI-powered generation (OpenAI), manual entry. User authentication included.
|
||||
|
||||
Developer: Vadym Samoilenko | License: Corporate — Oliver Marketing
|
||||
Oliver Metadata Tool v3.1 is a Flask/FastAPI-based web application for universal file metadata management and creation. It supports 300+ file formats through ExifTool integration and enables metadata ingestion from four sources: file imports (CSV/Excel/JSON), AI generation via OpenAI, manual entry, and reusable templates. The tool targets enterprise users who need to batch-process metadata across heterogeneous file types with audit logging, user authentication (local + Azure AD SSO), and AI-powered intelligent metadata generation with token tracking.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** HTML/JS
|
||||
- **Backend:** Python + FastAPI
|
||||
- **AI:** OpenAI
|
||||
- **Auth:** Built-in
|
||||
- **Infrastructure:** Docker + docker-compose
|
||||
|
||||
- **Frontend:** Jinja2 templates, HTML/CSS/JavaScript, MSAL.js (Azure AD SSO)
|
||||
- **Backend:** FastAPI 0.109.0+, Flask 2.3.0+, Python 3.8+
|
||||
- **Database:** SQLite (oliver_metadata.db)
|
||||
- **Infrastructure:** Docker, Docker Compose, Apache reverse proxy (subpath routing)
|
||||
- **AI/ML:** OpenAI GPT-4o-mini (configurable), tiktoken for token counting, tenacity for retry logic
|
||||
- **Key libraries:** PyExifTool 0.5.6+, pandas, openpyxl, pdfplumber, Pillow, pytesseract, mutagen, msal (for Azure AD)
|
||||
|
||||
## Architecture
|
||||
|
||||
The application uses a hybrid FastAPI/Flask architecture with the following components:
|
||||
|
||||
- **Web Framework:** Primary entry via `web_app.py` (Flask); core API likely in FastAPI
|
||||
- **Metadata Engine:** ExifTool subprocess integration via PyExifTool for reading/writing 300+ formats
|
||||
- **File Processing Pipeline:**
|
||||
- CSV/Excel/JSON import → pandas DataFrames → smart column mapping (fuzzy matching)
|
||||
- Image processing: Pillow, pytesseract (OCR), piexif (EXIF)
|
||||
- PDF: pdfplumber, PyPDF2, pdf2image for text extraction
|
||||
- Video: mutagen, ffmpeg-python, pymediainfo
|
||||
- Office docs: python-docx, python-pptx
|
||||
- **AI Generation:** OpenAI API calls with template variable substitution ({filename}, {date}, {user}, custom)
|
||||
- **Authentication:**
|
||||
- Local: username/password with Werkzeug password hashing
|
||||
- SSO: Azure AD via MSAL (client-side MSAL.js in browser, server-side MSAL library)
|
||||
- **Database:** SQLite with user management, audit logging, session tracking, AI usage metrics
|
||||
- **Deployment:** Docker Compose with persistent volumes (uploads, database, output), Apache reverse proxy at root path `/solventum-image-metadata`
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Browser / MSAL.js (Azure AD SSO) │
|
||||
└────────────────────┬────────────────────────────────────┘
|
||||
│ HTTPS (Apache reverse proxy)
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Apache (ai-sandbox.oliver.solutions) │
|
||||
│ └─► /solventum-image-metadata ──► localhost:5001 │
|
||||
└────────────────────┬────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────┐
|
||||
│ Flask/FastAPI Server │
|
||||
│ (web_app.py) │
|
||||
│ :5001 in container │
|
||||
└────┬──────────────┬────┘
|
||||
│ │
|
||||
┌──────▼──┐ ┌─────▼─────────┐
|
||||
│ SQLite │ │ ExifTool │
|
||||
│ Database│ │ (subprocess) │
|
||||
│ (auth, │ │ Metadata I/O │
|
||||
│ audit) │ │ 300+ formats │
|
||||
└─────────┘ └─────┬─────────┘
|
||||
│
|
||||
┌──────▼────────┐
|
||||
│ File Pipeline│
|
||||
│ (PIL, OCR, │
|
||||
│ pandas, etc.)│
|
||||
└─────┬────────┘
|
||||
│
|
||||
┌─────▼──────┐
|
||||
│ OpenAI API │
|
||||
│ (GPT-4o) │
|
||||
└─────────────┘
|
||||
```
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
# Setup
|
||||
python3 -m venv venv_local
|
||||
source venv_local/bin/activate
|
||||
pip install -r requirements.txt
|
||||
|
||||
# System dependencies (macOS)
|
||||
brew install exiftool tesseract tesseract-lang poppler
|
||||
|
||||
# System dependencies (Linux)
|
||||
sudo apt-get install libimage-exiftool-perl tesseract-ocr tesseract-ocr-chi-sim tesseract-ocr-chi-tra tesseract-ocr-jpn tesseract-ocr-kor poppler-utils
|
||||
|
||||
# Verify ExifTool
|
||||
exiftool -ver # Should be 12.15+
|
||||
|
||||
# Run locally
|
||||
python web_app.py
|
||||
# Opens http://localhost:5001 automatically
|
||||
|
||||
# Docker
|
||||
docker-compose up -d
|
||||
# Access at http://localhost:5001
|
||||
|
||||
# Database initialization (manual)
|
||||
python -c "from src.database import Database; db = Database(); print('Database initialized')"
|
||||
|
||||
# Create user (interactive)
|
||||
python
|
||||
>>> from src.database import Database
|
||||
>>> db = Database()
|
||||
>>> db.create_user(username='newuser', password='password123', email='user@example.com', full_name='New User', auth_method='local')
|
||||
|
||||
# List users
|
||||
python
|
||||
>>> from src.database import Database
|
||||
>>> db = Database()
|
||||
>>> users = db.get_all_users()
|
||||
>>> for user in users: print(f"{user['username']} - Last login: {user['last_login']}")
|
||||
|
||||
# Test imports
|
||||
python -c "from src.database import Database; from src.config import Config; print('✅ All imports successful')"
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `docker compose up --build`
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/solventum-image-metadata`
|
||||
|
||||
- **Server:** ai-sandbox.oliver.solutions
|
||||
- **Deploy:** `docker-compose up -d` (or `./docker-run.sh start`)
|
||||
- **URL:** https://ai-sandbox.oliver.solutions/solventum-image-metadata
|
||||
- **Port:** 5001 (inside container; reverse-proxied via Apache)
|
||||
- **Service:** Docker Compose (systemd service not documented)
|
||||
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/solventum-image-metadata
|
||||
|
||||
**Deployment Notes:**
|
||||
- Application runs in Docker with three persistent volumes: `uploads`, `database`, `output`
|
||||
- Apache reverse proxy must match `ROOT_PATH=/solventum-image-metadata` (no trailing slash)
|
||||
- REDIRECT_URI in `.env` must exactly match Azure AD App Registration (includes `/auth/callback`)
|
||||
- Health check: `curl http://localhost:5001/login` every 30s
|
||||
- Restart policy: `unless-stopped`
|
||||
|
||||
## Environment Variables
|
||||
|
||||
- `SECRET_KEY` — Flask secret key for session management; generate with `python3 -c "import secrets; print(secrets.token_hex(32))"`
|
||||
- `DOCKER_MODE` — Set to `true` when running in Docker
|
||||
- `ROOT_PATH` — Subpath prefix for Apache reverse proxy (e.g., `/solventum-image-metadata`); must match reverse proxy config
|
||||
- `AZURE_TENANT_ID` — Azure AD tenant ID (e.g., `e519c2e6-bc6d-4fdf-8d9c-923c2f002385`)
|
||||
- `AZURE_CLIENT_ID` — Azure AD application client ID (e.g., `9079054c-9620-4757-a256-23413042f1ef`)
|
||||
- `AZURE_CLIENT_SECRET` — Azure AD client secret from Certificates & secrets (required for server-side MSAL flow)
|
||||
- `REDIRECT_URI` — OAuth2 redirect URI; must match Azure App Registration exactly (e.g., `https://ai-sandbox.oliver.solutions/solventum-image-metadata/auth/callback`)
|
||||
- `ALLOWED_TENANT_IDS` — Comma-separated list of allowed Azure tenant IDs; leave empty to allow any organizational tenant
|
||||
- `OPENAI_API_KEY` — OpenAI API key for AI metadata generation (optional)
|
||||
- `AI_MODEL` — OpenAI model (default: `gpt-4o-mini`)
|
||||
- `MAX_TOKENS` — Max tokens for AI responses (default: `500`)
|
||||
- `TEMPERATURE` — AI temperature parameter (default: `0.5`)
|
||||
- `SUPERADMIN_EMAIL` — Email auto-created as admin on first startup via SSO (e.g., `vadymsamoilenko@oliver.agency`)
|
||||
- `ENABLE_TEST_USER` — Enable test user account (default: `false`)
|
||||
- `HTTPS_ONLY` — Enforce HTTPS (default: `true`)
|
||||
- `DEBUG` — Enable Flask debug mode (default: `false`)
|
||||
|
||||
## API / Endpoints
|
||||
|
||||
Not fully documented in provided files. Known endpoints:
|
||||
- `GET /login` — Login page
|
||||
- `POST /auth/callback` — Azure AD OAuth2 callback
|
||||
- Metadata import endpoints (file upload, AI generation, manual entry, templates)
|
||||
- Metadata export/CSV endpoints
|
||||
- User management endpoints (for admin)
|
||||
|
||||
## Known Issues
|
||||
|
||||
- **Azure AD SSO:** Fixed in commit `0976ee9` — AADSTS900144 client_id error resolved
|
||||
- **Filename collisions:** Fixed in commit `1156209` — prevented collisions breaking Excel metadata lookup
|
||||
- **Background AI processing:** Implemented in commit `ebc2322` for bulk uploads (up to 100 files) with polling
|
||||
- **MSAL.js loading:** Fixed in commits `ff3b89f`, `eaa12be`, `154658f` — async initialization and CDN migration to jsdelivr
|
||||
- **
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
|
|
@ -35,4 +187,4 @@ Developer: Vadym Samoilenko | License: Corporate — Oliver Marketing
|
|||
## Change Log
|
||||
| Date | Requested | Changed | Files |
|
||||
|------|-----------|---------|-------|
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
|
|
@ -270,6 +270,9 @@ Pipeline phases:
|
|||
## Change Log
|
||||
| Date | Requested | Changed | Files |
|
||||
|------|-----------|---------|-------|
|
||||
| 2026-04-29 | Plugin installation | Added levnikolaevich/claude-code-skills, codebase-audit-suite, agile-workflow | Claude Code configuration |
|
||||
| 2026-04-29 | Search and setup | Repository found, plugin path identified | No files changed |
|
||||
| 2026-04-29 | Code testing plugins | Plugin repository reference, PR review toolkit details | anthropics/claude-plugins-official |
|
||||
| 2026-04-29 | Cloud Code skills research | GitHub search for testing tools, verified references | None |
|
||||
| 2026-04-29 | Log | Glossary feature implementation | Added locale registry, Pydantic models, services for parsing/retrieval/embedding | lib/locales.py, models/glossary.py, services/glossary_service.py, services/embedding_service.py |
|
||||
| 2026-04-29 | Glossary integration | Added app/lib/ exception to gitignore, reviewed translation table structure | .gitignore |
|
||||
|
|
|
|||
|
|
@ -1,9 +1,9 @@
|
|||
---
|
||||
name: "WSJ File Naming Tool"
|
||||
client: Dow Jones / WSJ
|
||||
client: Dow Jones
|
||||
status: active
|
||||
server: optical-dev
|
||||
tech: [Python, HTML/JS, Google Gemini]
|
||||
server: local
|
||||
tech: [PHP, JavaScript, HTML/CSS, Google Gemini API, Microsoft MSAL, CSV]
|
||||
local_path: /Users/ai_leed/Documents/Projects/Oliver/wsj-filenaming
|
||||
deploy: python server.py
|
||||
url:
|
||||
|
|
@ -12,20 +12,91 @@ created: 2026-04-14
|
|||
---
|
||||
|
||||
## Overview
|
||||
Standardized job naming tool for Dow Jones naming convention. AI command bar (Gemini) + interactive spreadsheet UI.
|
||||
|
||||
**Format:** `[OMGID] - [Domain]-[Subteam]-[Brand]-[Initiative]-[YY]-[Seq]_[AssetName]_v[Version]`
|
||||
wsj-filenaming is a web-based tool for generating standardized job names following Dow Jones' naming convention. It serves marketing and brand teams who need to create compliant filenames across multiple domains (Performance Marketing, Brand, Events, B2B). The tool features an AI-powered command bar using Google Gemini for natural language job creation, an interactive spreadsheet interface with live filename generation, dropdown validation, and Azure AD SSO authentication via MSAL.
|
||||
|
||||
## Tech Stack
|
||||
- **Frontend:** Vanilla JS + HTML
|
||||
- **Backend:** Python
|
||||
- **AI:** Google Gemini
|
||||
- **Infrastructure:** No Docker
|
||||
- **Frontend:** Vanilla JavaScript, HTML5, CSS3, Google Sheets-like grid UI
|
||||
- **Backend:** PHP 7.4+
|
||||
- **Database:** N/A (file-based or session storage)
|
||||
- **Infrastructure:** Local web server (Apache/Nginx compatible)
|
||||
- **AI/ML:** Google Gemini API (command bar)
|
||||
- **Authentication:** Microsoft Azure AD / MSAL 2.32.2 (SSO)
|
||||
- **Key libraries:** MSAL.js (2.32.2), Google Gemini SDK
|
||||
|
||||
## Architecture
|
||||
The application follows a client-server model:
|
||||
|
||||
**Frontend Layer:**
|
||||
- Interactive spreadsheet grid with editable cells
|
||||
- Real-time filename generation as user edits fields
|
||||
- AI command bar powered by Google Gemini for natural language queries
|
||||
- Voice input support (speech-to-text)
|
||||
- Dropdown validation for Domain, Subteam, Brand, and Event fields
|
||||
- Campaign management (save/load/export CSV)
|
||||
- One-click filename copy functionality
|
||||
|
||||
**Backend Layer:**
|
||||
- PHP scripts for server-side logic (validation, CSV export, campaign persistence)
|
||||
- Azure AD / MSAL SSO authentication flow
|
||||
|
||||
**Data Flow:**
|
||||
```
|
||||
User Input (Grid/Command Bar)
|
||||
↓
|
||||
JavaScript validation + AI processing (Gemini API)
|
||||
↓
|
||||
Filename generation (client-side)
|
||||
↓
|
||||
PHP backend (export/save operations)
|
||||
↓
|
||||
Output (Grid display, CSV, clipboard)
|
||||
```
|
||||
|
||||
**Naming Convention Logic:**
|
||||
- Standard format: `[OMGID] - [Domain]-[Subteam]-[Brand]-[Initiative]-[YY]-[Sequence]_[AssetName]_v[Version]`
|
||||
- Event format: `[OMGID] - EVNT-[EventAbbrev]-[YY]-[Sequence]_[AssetName]_v[Version]`
|
||||
- Validation enforced via dropdown menus for all enumerated fields
|
||||
|
||||
## Dev Commands
|
||||
```bash
|
||||
# Start local development server (Apache/Nginx)
|
||||
# Ensure PHP 7.4+ is running and document root points to project directory
|
||||
php -S localhost:8000
|
||||
|
||||
# No build step required; edit files directly and reload browser
|
||||
|
||||
# Export/test functionality (via web interface)
|
||||
# Command bar: type natural language job descriptions
|
||||
# Manual grid editing: click cells and update values
|
||||
```
|
||||
|
||||
## Deployment
|
||||
- **Run:** `python server.py`
|
||||
- **Server:** local (intended for internal Dow Jones deployment)
|
||||
- **Deploy:** N/A (self-contained PHP + JS, no build pipeline)
|
||||
- **URL:** Determined by hosting setup (typically company intranet)
|
||||
- **Port:** Standard web server port (80/443)
|
||||
- **Service:** N/A (runs on existing web server)
|
||||
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/wsj-filenaming`
|
||||
|
||||
## Environment Variables
|
||||
- `GEMINI_API_KEY` — Google Gemini API key for command bar AI functionality
|
||||
- `AZURE_CLIENT_ID` — Azure AD application ID for MSAL SSO
|
||||
- `AZURE_TENANT_ID` — Azure AD tenant ID
|
||||
- `AZURE_REDIRECT_URI` — OAuth redirect URI (must match registered URI in Azure; note: trailing slash fix applied in commit b13985e)
|
||||
|
||||
## Known Issues
|
||||
- **MSAL Authentication:**
|
||||
- AADSTS50011 error fixed by ensuring redirect URI trailing slash matches Azure registration (commit b13985e)
|
||||
- AADSTS90014 error fixed by enabling `storeAuthStateInCookie` (commit cee80cc)
|
||||
- MSAL CDN pinned to v2.32.2 (known stable; v2.38.3 caused issues per commit 74ffaf5)
|
||||
- `msalInstance.initialize()` must be called before MSAL API calls (commit f4b4630)
|
||||
- **README incomplete:** Setup section cut off mid-sentence in source file
|
||||
- **No documented test suite or CI/CD pipeline**
|
||||
|
||||
## Git
|
||||
- **Remote:** `git@bitbucket.org:zlalani/wsj-filenaming.git`
|
||||
- **Recent work:** Azure AD MSAL SSO integration (commits f4b4630–523be05), Builder/Decoder page rebuild (1b2cfb8)
|
||||
|
||||
## Sessions
|
||||
### 2026-04-14 – Project catalogued
|
||||
**Done:** Added to Obsidian second brain.
|
||||
|
|
@ -34,4 +105,4 @@ Standardized job naming tool for Dow Jones naming convention. AI command bar (Ge
|
|||
## Change Log
|
||||
| Date | Requested | Changed | Files |
|
||||
|------|-----------|---------|-------|
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
| 2026-04-14 | Initial setup | Note created | — |
|
||||
|
|
@ -170,3 +170,15 @@ tags: [daily]
|
|||
- 13:10 | `video-accessibility`
|
||||
- **Asked:** Search GitHub for Cloud Code skills for testing and checking code and applications.
|
||||
- **Done:** Checked key references to verify Cloud Code testing skills and resources.
|
||||
- 13:13 | `video-accessibility`
|
||||
- **Asked:** Search GitHub for Claude code testing and review skills for cloud code applications.
|
||||
- **Done:** Provided official Anthropic plugin repository details and listed available code review and testing plugins.
|
||||
- 13:14 | `video-accessibility`
|
||||
- **Asked:** Asked | Searched GitHub for Claude Code skills plugins for testing and code verification
|
||||
- **Done:** Done | Located repository and identified installation command for Claude Code skills plugin
|
||||
- 13:18 | `video-accessibility`
|
||||
- **Asked:** Search GitHub for Claude Code skills for testing and code verification.
|
||||
- **Done:** Found and documented installation of claude-code-skills plugin with codebase-audit-suite and agile-workflow plugins.
|
||||
- 13:23 (<1min) | `Oliver (root)`
|
||||
- **Asked:** Check Obsidian integration as a second brain and ensure complete project information is captured for developers.
|
||||
- **Done:** Updated 28 projects with complete metadata (stack, deployment, URLs, changelogs) and fixed the outdated Obsidian Second Brain project documentation.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue