diff --git a/01 Projects/Barclays-banner-builder/Barclays Banner Builder.md b/01 Projects/Barclays-banner-builder/Barclays Banner Builder.md index c5d4c0e..b1b38e3 100644 --- a/01 Projects/Barclays-banner-builder/Barclays Banner Builder.md +++ b/01 Projects/Barclays-banner-builder/Barclays Banner Builder.md @@ -23,6 +23,10 @@ created: 2026-04-17 - **Local path:** `/Volumes/SSD/Projects/Oliver/Barclays-banner-builder` ## Sessions +### 2026-04-17 – Create an idempotent deployment script for +**Asked:** Create an idempotent deployment script for Ubuntu Docker setup with Apache reverse proxy and database migrations. +**Done:** Analyzed project structure and created deployment script with Docker builds, database initialization, and migration handling. + ### 2026-04-17 – Create an idempotent deployment script for **Asked:** Create an idempotent deployment script for Docker containers on Ubuntu with Apache reverse proxy and database migrations. **Done:** Analyzed app data, designed application concept with deployment architecture, and created deployment script with Docker build, cache management, and database initialization. @@ -43,6 +47,7 @@ created: 2026-04-17 ## Change Log | Date | Requested | Changed | Files | |------|-----------|---------|-------| +| 2026-04-17 | Deployment script | Docker build caching, database migrations, idempotent logic | deploy.sh, architecture/_index.md | | 2026-04-17 | Deployment automation | Docker build script, database migrations, Apache reverse proxy config, CLAUDE.md documentation | deploy.sh, CLAUDE.md, docker-compose.yml | | 2026-04-17 | Deployment infrastructure | docker-compose.yml, docker-compose.prod.yml, Apache vhost config, .env.example | docker-compose.yml, docker-compose.prod.yml, apache/barclays-copygen.conf, .env.example | | 2026-04-17 | Deployment infrastructure | docker-compose.yml, docker-compose.prod.yml, apache/barclays-copygen.conf, .env.example | docker-compose.yml, docker-compose.prod.yml, apache/barclays-copygen.conf | diff --git a/99 Daily/2026-04-17.md b/99 Daily/2026-04-17.md index 6f990bb..7f99ed2 100644 --- a/99 Daily/2026-04-17.md +++ b/99 Daily/2026-04-17.md @@ -17,3 +17,6 @@ tags: [daily] - 11:55 | `Barclays-banner-builder` - **Asked:** Create an idempotent deployment script for Docker containers on Ubuntu with Apache reverse proxy and database migrations. - **Done:** Analyzed app data, designed application concept with deployment architecture, and created deployment script with Docker build, cache management, and database initialization. +- 12:00 (1min) | `Barclays-banner-builder` + - **Asked:** Create an idempotent deployment script for Ubuntu Docker setup with Apache reverse proxy and database migrations. + - **Done:** Analyzed project structure and created deployment script with Docker builds, database initialization, and migration handling. diff --git a/wiki/architecture/_index.md b/wiki/architecture/_index.md index f3067f4..6fd2acc 100644 --- a/wiki/architecture/_index.md +++ b/wiki/architecture/_index.md @@ -27,3 +27,6 @@ Cross-cutting architectural decisions that appear in multiple Oliver projects. 3. **AI pre-structuring before RAG indexing** — improves retrieval quality 4. **Hotfolder + archive pattern** — prevents reprocessing in Box automations 5. **DEV_AUTH_BYPASS** — skip Azure AD in local dev, always use real auth in production + + +| [[wiki/architecture/optical-dev-server-deploy\|optical-dev-server-deploy]] | optical-dev GCP server: single-vhost Apache, Include pattern, port table, deploy script cache | Barclays Banner Builder, all Oliver projects | \ No newline at end of file diff --git a/wiki/architecture/optical-dev-server-deploy.md b/wiki/architecture/optical-dev-server-deploy.md new file mode 100644 index 0000000..abe68aa --- /dev/null +++ b/wiki/architecture/optical-dev-server-deploy.md @@ -0,0 +1,270 @@ +--- +title: "optical-dev Server — Apache Deployment Pattern" +description: "Single-vhost Apache pattern on optical-dev.oliver.solutions GCP server — port allocation, Include fragments, SPA routing, deploy script best practices" +tags: [architecture, apache, deployment, docker, ubuntu, gcp] +created: 2026-04-17 +updated: 2026-04-17 +--- + +# optical-dev Server — Apache Deployment Pattern + +## Server Profile + +| Field | Value | +|-------|-------| +| Hostname | optical-dev.oliver.solutions | +| SSH alias | `optical-dev` (see `~/.ssh/config`) | +| OS | Ubuntu 24.04 LTS | +| Cloud | GCP europe-west2-b | +| Web server | Apache 2.4.58 | +| Docker | 29.3.0 | +| Node.js | v22.22.2 | +| npm | 10.9.7 | +| Python | 3.12.3 | + +--- + +## Apache Single-Vhost Pattern + +**One vhost file handles ALL projects:** +``` +/etc/apache2/sites-available/optical-dev.oliver.solutions.conf +``` + +Each project is included as a fragment: +```apache + + ServerName optical-dev.oliver.solutions + ... + Include /opt/project-a/deploy/apache-project-a.conf + Include /opt/project-b/deploy/apache-project-b.conf + Include /opt/barclays-banner-builder/deploy/apache-barclays.conf + +``` + +Deploy scripts inject the Include line automatically via sed (idempotent): +```bash +INCLUDE_LINE=" Include /opt/barclays-banner-builder/deploy/apache-barclays.conf" +if ! sudo grep -qF "" ""; then + sudo sed -i "s|| +|" "" + sudo apache2ctl configtest && sudo systemctl reload apache2 +fi +``` + +--- + +## Apache Fragment Pattern + +### SPA (React/Vue) with API backend +```apache +# API proxy — strip project prefix so FastAPI sees /api/... +ProxyPass /my-project/api/ http://127.0.0.1:PORT/api/ +ProxyPassReverse /my-project/api/ http://127.0.0.1:PORT/api/ + +# Swagger docs +ProxyPass /my-project/docs http://127.0.0.1:PORT/docs +ProxyPassReverse /my-project/docs http://127.0.0.1:PORT/docs + +# SPA static files +Alias /my-project /var/www/html/my-project + + Options -Indexes +FollowSymLinks + AllowOverride None + Require all granted + + RewriteEngine On + RewriteBase /my-project/ + + # Pass real files/dirs through + RewriteCond %{REQUEST_FILENAME} -f [OR] + RewriteCond %{REQUEST_FILENAME} -d + RewriteRule ^ - [L] + + # Everything else → index.html (React Router handles client-side nav) + RewriteRule ^ index.html [L] + +``` + +### Key rules +- MUST come BEFORE — Apache processes top to bottom +- must include trailing slash +- FollowSymLinks is required for symlinked static assets (e.g., illustrations folder) +- Apache serves SPA directly from `/var/www/html/` — no Nginx container needed in prod + +--- + +## Port Allocation Table + +### Occupied (do NOT use) + +| Port | Service | +|------|--------| +| 3000 | Node app | +| 3001 | Node app | +| 3050 | Node app | +| 3456 | Node app | +| 5137 | Vite dev | +| 5491 | Python app | +| 5492 | Python app | +| 8000 | OliVAS FastAPI | +| 8001 | FastAPI | +| 8002 | FastAPI | +| 8040 | FastAPI | +| 8800 | App | +| 9000 | App | +| 20201 | App | +| 20202 | App | +| 27017 | MongoDB | +| 6389 | Redis (custom port) | +| 6379 | Redis (standard — likely used) | + +### Allocated + +| Port | Project | +|------|--------| +| 8010 | Barclays Banner Builder API | + +### Available range +When adding new projects, choose ports in ranges: **8011–8039**, **8041–8799**, **8801–8999** + +--- + +## File System Layout + +| Path | Purpose | +|------|--------| +| `/opt//` | Git repo root for all projects | +| `/var/www/html//` | Apache-served static files (React/Vue dist) | +| `/etc/apache2/sites-available/optical-dev.oliver.solutions.conf` | Single vhost config | +| `/opt//deploy/apache-.conf` | Per-project Apache Include fragment | +| `/opt//.deploy_state/` | Build cache hashes (dockerfile_hash, npm_hash) | +| `/opt//.env` | Production secrets (not in git) | + +--- + +## Deploy Script Patterns + +### 1. Hash-based build cache (avoid redundant rebuilds) +```bash +# Skip Docker image rebuild if Dockerfile + pyproject unchanged +DOCKERFILE_HASH=$(md5sum backend/Dockerfile pyproject.toml 2>/dev/null | md5sum | cut -d' ' -f1) +LAST_HASH_FILE=".deploy_state/dockerfile_hash" + +if [[ -f "$LAST_HASH_FILE" ]] && [[ "$(cat "$LAST_HASH_FILE")" == "$DOCKERFILE_HASH" ]]; then + echo "Dockerfile unchanged — skipping rebuild" +else + docker compose -f docker-compose.prod.yml build --parallel api worker + echo "$DOCKERFILE_HASH" > "$LAST_HASH_FILE" +fi + +# Same pattern for npm packages +PKG_HASH=$(md5sum package.json package-lock.json 2>/dev/null | md5sum | cut -d' ' -f1) +``` + +### 2. First-run detection via SQL COUNT +```bash +# Idempotent — only seeds if table is empty +USER_COUNT=$(docker compose exec -T api python -c " +import asyncio +from app.database import AsyncSessionLocal +from sqlalchemy import text +async def count(): + async with AsyncSessionLocal() as db: + r = await db.execute(text('SELECT COUNT(*) FROM users')) + print(r.scalar()) +asyncio.run(count()) +" 2>/dev/null || echo "0") + +if [[ "$USER_COUNT" -eq "0" ]]; then + docker compose exec -T api python scripts/seed_admin.py +fi +``` + +### 3. Postgres readiness check (before migrations) +```bash +for i in $(seq 1 30); do + if docker compose exec -T postgres pg_isready -U "${POSTGRES_USER}" &>/dev/null; then + break + fi + [[ $i -eq 30 ]] && { echo "Postgres not ready"; exit 1; } + sleep 1 +done +``` + +### 4. Frontend deploy (npm build → rsync to Apache dir) +```bash +cd frontend +VITE_BASE_PATH="/my-project" npm run build +cd .. + +sudo mkdir -p /var/www/html/my-project +sudo find /var/www/html/my-project -mindepth 1 -not -name 'illustrations' -delete +sudo cp -r frontend/dist/. /var/www/html/my-project/ +sudo chmod -R a+rX /var/www/html/my-project +``` + +### 5. Symlink large static assets instead of copying +```bash +# Avoids copying hundreds of MB of illustrations on every deploy +if [[ ! -L "/var/www/html/my-project/illustrations" ]]; then + sudo ln -sfn "/opt/my-project/assets/illustrations" "/var/www/html/my-project/illustrations" +fi +``` + +--- + +## Vite Subdirectory Configuration + +When React app lives at `/project/` (not root `/`): + +**vite.config.ts** +```ts +export default defineConfig({ + base: process.env.VITE_BASE_PATH ?? "/", +}) +``` + +**main.tsx** +```tsx +const basename = import.meta.env.VITE_BASE_PATH ?? "/"; + +``` + +**api/client.ts** +```ts +const API_PREFIX = import.meta.env.VITE_BASE_PATH ?? ""; +// All fetch() calls: fetch(`${API_PREFIX}/api/...`) +``` + +Build command: `VITE_BASE_PATH=/my-project npm run build` + +--- + +## FastAPI Behind Apache Proxy + +```bash +# uvicorn flags required for correct IP forwarding behind Apache/LB +uvicorn app.main:app \ + --host 0.0.0.0 \ + --port 8000 \ + --workers 2 \ + --proxy-headers \ + --forwarded-allow-ips='*' +``` + +In docker-compose: bind only to `127.0.0.1::8000` — never expose containers directly. + +--- + +## No WebSocket Rule + +Apache + corporate LB timeout is ~30–60s. Solution: HTTP job polling. + +``` +POST /api/jobs/generate → 202 { job_id } + ↓ client polls every 2s +GET /api/jobs/{id} → { status: pending|running|done, result? } +``` + +See [[wiki/architecture/gcp-deployment-lb-timeout|gcp-deployment-lb-timeout]] for full pattern. \ No newline at end of file