dow-prod-tracker/DEPLOY.md
DJP 7e7ef7b7c1 deploy.sh: auto-detect free host ports, render Apache conf per-deploy
You were right — everything's containerized, the host ports are just
reverse-proxy targets (+ an optional psql peephole for the db). Hardcoding
them is why the local smoke test face-planted on 5492 (amazon-transcreation
was squatting it) and would have done the same any time anything else
bound :3002 or :5492 on the shared server.

docker-compose.yml:
- ports now reference `${APP_HOST_PORT:-3002}` and `${DB_HOST_PORT:-5492}`.
  Defaults match the prior-committed values; override via env vars.
  Container-internal ports (3000, 5432) never change.

apache/dow-prod-tracker.conf → .conf.tmpl:
- Moved to a committed template with `${APP_HOST_PORT}` placeholders in
  both the WebSocket rewrite and the ProxyPass/ProxyPassReverse lines.
- deploy.sh renders the real .conf from the template on every run with
  the chosen port substituted in. Rendered .conf is gitignored so it
  can vary per server without drift.

deploy.sh:
- New is_port_free() and find_free_port() using bash's /dev/tcp — no
  external tool dependency, works identically on Ubuntu and macOS.
- After `docker compose down` (which frees any of OUR ports), probe for
  APP_HOST_PORT starting from 3002 and DB_HOST_PORT from 5492. Pick the
  first free port (scan up to 50). Warn if the preferred port was busy.
  Honors explicit override: `APP_HOST_PORT=3005 ./deploy.sh` works.
- Exports the chosen ports before `docker compose up` so compose
  substitutes them into the `ports:` mappings.
- Renders apache/dow-prod-tracker.conf from the .tmpl with the same
  APP_HOST_PORT, every deploy. If the Apache Include line is already in
  the vhost, we reload Apache anyway (picks up the re-rendered snippet
  in case the port changed).
- Health check URL uses APP_HOST_PORT.
- "Deploy complete" banner now prints the chosen ports.

.gitignore:
- Added docker-compose.override.yml (per-machine local overrides) and
  apache/dow-prod-tracker.conf (rendered by deploy.sh, varies per server).

DEPLOY.md updated with the auto-detection behaviour and override recipe.

Sanity-checked locally:
- is_port_free correctly identifies 5492 busy (amazon-transcreation),
  5493 busy (our smoke-test db), 3002 busy (Docker Desktop grabs 3000-3002
  on this Mac), and picks 5494/3003 respectively.
- `APP_HOST_PORT=3999 DB_HOST_PORT=5999 docker compose config` produces
  published ports 3999 and 5999.
- `bash -n deploy.sh` clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 19:59:30 -04:00

8.2 KiB

Deploying dow-prod-tracker

Target: https://optical-dev.oliver.solutions/dow-prod-tracker, hosted on the shared Oliver Agency dev box alongside hp-prod-tracker.

Run the deploy script from the repo root on the server:

cd /srv/dow-prod-tracker          # or wherever you cloned it
./deploy.sh

The first run will install Docker, Compose, Apache and UFW if missing, prompt you to fill in .env from .env.example, then bring everything up. Subsequent runs are idempotent — safe to re-run for updates.


Why this deploy is different from the rest

The shared server runs multiple apps from sibling deploy directories. Two things must hold or those apps will silently corrupt each other (this has bitten us before — see CLAUDE.md):

  1. docker-compose.yml MUST pin a top-level name: field. We pin name: dow-prod-tracker. Without this, Compose defaults the project name to the parent directory name. If two apps both live under deploy/, they collapse onto the same project and fight over containers (deploy-db-1) and volumes (deploy_pgdata). Deploying one would silently evict the other and destroy its data.

  2. Every docker compose invocation in deploy.sh passes -p dow-prod-tracker as belt-and-braces. This is redundant with name: today, but if anyone moves the name: line out of the compose file, or runs commands by hand from a different cwd, the -p flag is the safety net.

The deploy script also enforces:

Concern Value Where
Compose project name dow-prod-tracker name: in docker-compose.yml + -p in deploy.sh
App port (host) 3002 preferred, auto-bumped if busy deploy.sh probes, exports APP_HOST_PORT, renders apache/dow-prod-tracker.conf from the .tmpl
DB port (host) 5492 preferred, auto-bumped if busy deploy.sh probes, exports DB_HOST_PORT into compose
DB name dow_prod_tracker docker-compose.yml env + .env DATABASE_URL
App URL path /dow-prod-tracker next.config.ts basePath
Apache reverse proxy → 127.0.0.1:${APP_HOST_PORT}/dow-prod-tracker rendered on each deploy from apache/dow-prod-tracker.conf.tmpl

Auto port selection. Everything runs in Docker — the container-internal ports (app 3000, db 5432) never change. The host ports are only used by Apache's reverse proxy (for the app) and direct psql debugging (for the db). If 3002 or 5492 happens to be in use (another app on the shared server, something on your laptop), deploy.sh picks the next free port upward and stitches it through: compose env var, Apache ProxyPass, health check, "Deploy complete" banner. No manual port negotiation.

To force specific ports, export them before running:

APP_HOST_PORT=3005 DB_HOST_PORT=5495 ./deploy.sh

hp-prod-tracker lives on 3001. Dow prefers 3002; the probe will skip 3001 if something (like HP) is on it.


.env.production checklist

Required env vars on the server:

DATABASE_URL=postgresql://postgres:<DB_PASSWORD>@db:5432/dow_prod_tracker?schema=public
DB_PASSWORD=<long random>
AUTH_SECRET=<openssl rand -base64 32>

# Auth — local credentials in MVP. Flip to true post-MVP once Entra is wired.
NEXT_PUBLIC_AUTH_ENTRA_ENABLED=false

# Entra (only required when NEXT_PUBLIC_AUTH_ENTRA_ENABLED=true)
AZURE_CLIENT_ID=
AZURE_TENANT_ID=
AZURE_REDIRECT_URI=https://optical-dev.oliver.solutions/dow-prod-tracker/login

# OMG webhook receiver
OMG_WEBHOOK_SECRET=<openssl rand -hex 32>     # share with Shashank
OMG_WEBHOOK_ALLOW_INSECURE=false

# AI chat (optional — chat features degrade gracefully if absent)
ANTHROPIC_API_KEY=
OLLAMA_HOST=http://10.24.42.219:11434
OLLAMA_CHAT_MODEL=gemma4:latest
OLLAMA_EMBED_MODEL=nomic-embed-text

# Cron + API key (used for scheduled tasks + machine-to-machine API access)
CRON_SECRET=<openssl rand -hex 32>
API_KEY=<openssl rand -hex 32>

# Initial admin (optional — overrides the default admin@dowjones.com / random pw)
DOW_ADMIN_EMAIL=admin@dowjones.com
DOW_ADMIN_PASSWORD=                            # leave blank → seed prints a random one

First-time setup on a fresh server

# 1. Clone
sudo mkdir -p /srv && sudo chown $USER:$USER /srv
cd /srv
git clone git@bitbucket.org:zlalani/dow-prod-tracker.git
cd dow-prod-tracker

# 2. Configure
cp .env.example .env
$EDITOR .env                                    # fill in real secrets

# 3. Deploy (installs deps, builds, runs migrations, configures Apache + UFW)
./deploy.sh

# 4. Seed the Dow tenant + initial admin (one-time)
docker compose -p dow-prod-tracker exec app npm run db:seed
# → save the printed admin email + temp password

Updating an existing deployment

cd /srv/dow-prod-tracker
./deploy.sh                                     # pulls, rebuilds, restarts

The Prisma migration in prisma/migrations/20260420000000_init/ runs on container startup (via the standalone Next.js entrypoint). New migrations land automatically.


Verification — must pass before declaring deploy done

  1. Volume isolation — both apps' volumes must be distinct:

    docker volume ls | grep -E "hp-prod-tracker|dow-prod-tracker"
    

    Expect to see hp-prod-tracker_pgdata, hp-prod-tracker_uploads_data, dow-prod-tracker_pgdata, dow-prod-tracker_uploads_data — four distinct volumes.

  2. HP unaffected — hit HP's URL in a browser and confirm it loads. If it fails, the new deploy collided with it (don't continue — fix first).

  3. Health checkcurl https://optical-dev.oliver.solutions/dow-prod-tracker/api/health should return 200.

  4. Login — open the URL in a browser, sign in with the seed admin credentials, get redirected to /change-password?first=1, set a real password, land on /dashboard.

  5. XLSX ingest — upload Dow Jones_Studio Tracker_Example_*.xlsx via POST /api/projects/bulk-import?commit=false (curl or via the UI button). Expect a preview JSON with normalized rows + any row errors. Then commit with ?commit=true and spot-check a project in psql:

    docker compose -p dow-prod-tracker exec db \
      psql -U postgres -d dow_prod_tracker \
      -c "select \"omgJobNumber\", name, status, \"clientTeamId\" from projects limit 5;"
    
  6. OMG webhook — when Shashank confirms the payload shape, share OMG_WEBHOOK_SECRET and have OMG POST to https://optical-dev.oliver.solutions/dow-prod-tracker/api/webhooks/omg with header X-OMG-Signature: sha256=<hex hmac of body>.

  7. Per-team visibility — invite a test user as CLIENT_VIEWER, assign them to one client team only (Settings → Client Teams), accept the invite link, sign in, confirm they only see that team's projects.


Rollback

If a deploy goes wrong:

cd /srv/dow-prod-tracker
git log --oneline -5                           # find previous good commit
git checkout <previous-commit>
./deploy.sh --skip-pull                        # rebuild from that commit

If the database itself is in a bad state (rare — Prisma migrations are forward-only), restore from the latest postgres dump in /srv/backups/ (set up your own cron for these — not yet automated).


Common issues

"port is already allocated" — very unlikely now; deploy.sh auto-picks a free port before docker compose up. If it still happens, something grabbed the port between the probe and the bind. Just rerun ./deploy.sh. If it's a stale dow-prod-tracker container from a botched deploy, docker compose -p dow-prod-tracker down will release it first.

Apache returns 502 — the app container isn't running or its health check is failing. docker compose -p dow-prod-tracker logs app --tail 50.

"Project with that OMG number already exists in a different organization" — a previous import landed projects under a different org (e.g. dev seed). Either delete those projects, or recreate the org with the canonical dowjones.com domain (the seed and webhook both look up the org by domain).

/api/webhooks/omg returns 401 — signature mismatch. Verify both ends share the exact OMG_WEBHOOK_SECRET and that OMG sends the header as X-OMG-Signature: sha256=<hex>. For local stub testing only, set OMG_WEBHOOK_ALLOW_INSECURE=true in .env and restart.