diff --git a/99 Daily/2026-04-18.md b/99 Daily/2026-04-18.md index 4ca8ca7..746eb65 100644 --- a/99 Daily/2026-04-18.md +++ b/99 Daily/2026-04-18.md @@ -374,3 +374,21 @@ tags: [daily] - 21:51 | `aimpress` - **Asked:** Help with Proxmox server installation and network configuration. - **Done:** Discussed bootable USB preparation and VM creation process. +- 21:55 (<1min) | `aimpress` + - **Asked:** How to set up and configure Proxmox on a newly purchased server with a bootable USB? + - **Done:** Created a Proxmox VM (kali-linux, ID 200) and provided instructions to start it via Proxmox UI console for installation. +- 21:57 | `aimpress` + - **Asked:** Help with Proxmox server installation and disk partitioning setup. + - **Done:** Guided the developer through disk partitioning selection for Proxmox VM installation. +- 21:58 | `aimpress` + - **Asked:** Developer asked for help setting up a Proxmox server and checking its network visibility and power consumption. + - **Done:** Identified that RAPL only measures CPU power (~3.4W idle) and provided estimated total server consumption (25–35W idle) with recommendation to use a smart power meter for accurate full-system monitoring. +- 21:58 | `aimpress` + - **Asked:** How to install and configure Proxmox on a new server with help? + - **Done:** Reviewed server power consumption metrics and recommended Kali Linux installation settings (Xfce with top10 tools). +- 21:59 | `aimpress` + - **Asked:** How to install and configure Proxmox on a new server using a bootable USB? + - **Done:** Reviewed Kali Linux desktop environment options (Xfce vs GNOME vs KDE) and recommended keeping Xfce for VM performance. +- 21:59 | `aimpress` + - **Asked:** How to help with Proxmox server setup and Kali Linux VM installation? + - **Done:** Provided guidance on desktop environment selection (recommended Xfce) and console access methods via Proxmox noVNC. diff --git a/wiki/_master-index.md b/wiki/_master-index.md index 7504c95..4822f0a 100644 --- a/wiki/_master-index.md +++ b/wiki/_master-index.md @@ -23,8 +23,8 @@ This 3-hop pattern works for hundreds of articles without vector search. | [[wiki/tech-patterns/_index\|tech-patterns/]] | Recurring tech stacks: FastAPI, React/Vite, Next.js, Azure AD, AI, Box, One2Edit | 9 | | [[wiki/architecture/_index\|architecture/]] | Cross-cutting architectural patterns: Docker Compose, multi-agent AI, GCP timeout, RAG, hotfolder | 5 | | [[wiki/client-knowledge/_index\|client-knowledge/]] | Per-client notes for Ford, H&M, L'Oréal (2+ projects each) | 3 | -| [[wiki/concepts/_index\|concepts/]] | Atomic knowledge extracted from Claude Code sessions | 4 | -| [[wiki/connections/_index\|connections/]] | Cross-cutting insights linking 2+ concepts | 1 | +| [[wiki/concepts/_index\|concepts/]] | Atomic knowledge extracted from Claude Code sessions | 15 | +| [[wiki/connections/_index\|connections/]] | Cross-cutting insights linking 2+ concepts | 2 | | [[wiki/qa/_index\|qa/]] | Filed answers to queries (saved with `--file-back`) | 0 | | [[wiki/homelab/_index\|homelab/]] | Self-hosted infra: Proxmox install, IOMMU/PCI passthrough, hypervisor setup, budget builds | 4 | | [[wiki/web-agency/_index\|web-agency/]] | AI-assisted website building & selling: Claude Code, Nanobanana 2, Kling, LaunchPath MCP | 1 | diff --git a/wiki/concepts/_index.md b/wiki/concepts/_index.md index 72f7782..b1a7bb2 100644 --- a/wiki/concepts/_index.md +++ b/wiki/concepts/_index.md @@ -9,6 +9,17 @@ | [[wiki/concepts/msal-vanilla-js-pkce]] | MSAL.js v5 UMD script-tag PKCE for SPAs without bundlers — Azure portal platform type, Node.js JWKS validation | daily/2026-04-15.md | 2026-04-15 | | [[wiki/concepts/shell-static-deploy-patterns]] | Shell deploy scripts for static frontends — cp -r vs glob, Apache reload requirement, setup vs deploy distinction | daily/2026-04-15.md | 2026-04-15 | | [[wiki/concepts/fish-shell-path-config]] | Fish shell PATH configuration — /usr/local/bin not in default PATH, fish_add_path, SSH aliases as Fish functions | daily/2026-04-15.md | 2026-04-15 | +| [[wiki/concepts/microsoft-graph-api-mailbox-migration]] | Graph API app-only (client credentials) flow for reading shared mailboxes — Mail.Read.All, EML download, pagination | daily/2026-04-16.md | 2026-04-16 | +| [[wiki/concepts/mailcow-maildir-import]] | Mailcow Maildir volume path, rsync import, chown 5000:5000, mandatory doveadm index — without indexing IMAP shows nothing | daily/2026-04-16.md | 2026-04-16 | +| [[wiki/concepts/fastapi-mongodb-role-migration]] | Adding UserRole enum + MongoDB schema validator migration + idempotent seed in FastAPI lifespan | daily/2026-04-16.md | 2026-04-16 | +| [[wiki/concepts/monorepo-deploy-script-pitfall]] | deploy.sh checking backend/.git / frontend/.git silently skips git pull in monorepos — fix: single git pull at root | daily/2026-04-16.md | 2026-04-16 | +| [[wiki/concepts/nextjs-basepath-auth-redirects]] | Next.js basePath requires all auth redirect strings to include the prefix — middleware and API routes don't auto-prepend | daily/2026-04-16.md | 2026-04-16 | +| [[wiki/concepts/python-service-deployment-dotenv]] | Python service deploy checklist — venv pip install after git pull, .env.example → .env for new config.py requirements | daily/2026-04-16.md | 2026-04-16 | +| [[wiki/concepts/openai-max-completion-tokens]] | OpenAI newer models reject `max_tokens` — must use `max_completion_tokens` in both raw API calls and internal wrappers | daily/2026-04-17.md | 2026-04-17 | +| [[wiki/concepts/fish-abbr-patterns]] | Fish `abbr` expands visibly (git/docker/uv), `alias` for transparent replacements (ls/cat); forgit conflict resolution | daily/2026-04-17.md | 2026-04-17 | +| [[wiki/concepts/bitbucket-mcp-atlassian]] | Bitbucket MCP via aashari/mcp-server-atlassian-bitbucket; GitHub MCP is GitHub-only; Atlassian API tokens replace App Passwords | daily/2026-04-17.md | 2026-04-17 | +| [[wiki/concepts/proxmox-mcp-server]] | canvrno/ProxmoxMCP is the best community Proxmox MCP server; install to global Claude Code config for any-project access | daily/2026-04-18.md | 2026-04-18 | +| [[wiki/concepts/proxmox-vm-management-methods]] | Proxmox VM creation approaches — SSH+qm (ad-hoc), REST API, Terraform, Ansible, MCP; when to use each | daily/2026-04-18.md | 2026-04-18 | diff --git a/wiki/concepts/bitbucket-mcp-atlassian.md b/wiki/concepts/bitbucket-mcp-atlassian.md new file mode 100644 index 0000000..31b7076 --- /dev/null +++ b/wiki/concepts/bitbucket-mcp-atlassian.md @@ -0,0 +1,93 @@ +--- +title: "Bitbucket MCP — Atlassian Server Setup" +aliases: [bitbucket-mcp, atlassian-mcp-bitbucket, mcp-bitbucket, atlassian-api-tokens] +tags: [mcp, bitbucket, atlassian, claude-code, git, api-tokens] +sources: + - "daily/2026-04-17.md" +created: 2026-04-17 +updated: 2026-04-17 +--- + +# Bitbucket MCP — Atlassian Server Setup + +The official GitHub MCP server (`@modelcontextprotocol/server-github`) only works with GitHub — not Bitbucket Cloud or Bitbucket Server. For Bitbucket, use `aashari/mcp-server-atlassian-bitbucket`. Both MCP servers can coexist in Claude Code's global config without conflicts. Bitbucket App Passwords are deprecated; use Atlassian API Tokens instead. + +## Key Points + +- **GitHub MCP ≠ Bitbucket MCP** — they are completely separate servers with separate auth; the GitHub MCP server does not support the Bitbucket API +- Recommended Bitbucket MCP: `aashari/mcp-server-atlassian-bitbucket` (Bitbucket Cloud; npm package) +- Bitbucket **App Passwords are deprecated** — replaced by Atlassian API Tokens created at `id.atlassian.com/manage-profile/security/api-tokens` +- GitHub MCP and Bitbucket MCP coexist in Claude Code's global `~/.claude/settings.json` without conflicts — they appear as separate tool namespaces +- Promote MCP servers to the **global** Claude Code config if you want them available in every project (not just the current working directory) + +## Details + +### Why GitHub MCP Doesn't Work for Bitbucket + +`@modelcontextprotocol/server-github` calls the GitHub REST API (`api.github.com`). Bitbucket Cloud uses a completely different API (`api.bitbucket.org/2.0`) with different auth headers, endpoint paths, and resource models. There is no compatibility layer — you need a separate MCP server implementation. + +### Choosing a Bitbucket MCP Server + +| Server | Platform | Language | Notes | +|--------|----------|----------|-------| +| `aashari/mcp-server-atlassian-bitbucket` | Bitbucket Cloud | TypeScript/npm | Best maintained as of 2026-04 | +| Atlassian official Jira/Confluence MCP | Jira + Confluence | — | Does not cover Bitbucket repos | + +For Bitbucket Cloud (bitbucket.org), `aashari/mcp-server-atlassian-bitbucket` is the practical choice. + +### Installation (Global Claude Code Config) + +```json +// ~/.claude/settings.json — global, fires from any project +{ + "mcpServers": { + "github": { + "command": "npx", + "args": ["-y", "@modelcontextprotocol/server-github"], + "env": { + "GITHUB_PERSONAL_ACCESS_TOKEN": "ghp_..." + } + }, + "bitbucket": { + "command": "npx", + "args": ["-y", "mcp-server-atlassian-bitbucket"], + "env": { + "ATLASSIAN_USERNAME": "your@email.com", + "ATLASSIAN_API_TOKEN": "ATATT3x..." + } + } + } +} +``` + +Both servers appear under separate names in Claude Code — `github__*` tools and `bitbucket__*` tools (or whatever the server declares). + +### Creating an Atlassian API Token + +Bitbucket App Passwords (`https://bitbucket.org/account/settings/app-passwords/`) are legacy and deprecated. The replacement: + +1. Go to `https://id.atlassian.com/manage-profile/security/api-tokens` +2. Click **Create API token** +3. Give it a label (e.g., `claude-code-mcp`) +4. Copy the token immediately — it's only shown once +5. Use as `ATLASSIAN_API_TOKEN` with your Atlassian account email as `ATLASSIAN_USERNAME` + +The same API token works for Bitbucket Cloud, Jira, and Confluence since they share the Atlassian identity platform. + +### Claude Code Plugin Scope + +Claude Code plugins (including code-review, feature-dev) can be scoped to: +- **Project** scope — only active in the current project directory +- **User (global)** scope — active in every project + +To promote a plugin to global scope: open Claude Code settings → Plugins → find the plugin → change scope to "User". This is the right choice for utility plugins like `code-review` and `feature-dev` that apply to any project. + +## Related Concepts + +- [[wiki/concepts/fish-abbr-patterns]] — Fish shell configuration session where MCP setup was done +- [[wiki/claude-code/_index]] — Claude Code MCP documentation, hooks, and plugin configuration +- [[wiki/connections/graph-api-vs-msal-app-vs-delegated]] — API auth patterns (App Passwords → API Tokens mirrors the broader shift to OAuth-based tokens) + +## Sources + +- [[daily/2026-04-17.md]] — GitHub MCP vs Bitbucket question resolved; `aashari/mcp-server-atlassian-bitbucket` selected; Atlassian API token created at id.atlassian.com; `code-review` and `feature-dev` plugins promoted to global scope; GitHub + Bitbucket MCP confirmed to coexist without conflicts diff --git a/wiki/concepts/fastapi-mongodb-role-migration.md b/wiki/concepts/fastapi-mongodb-role-migration.md new file mode 100644 index 0000000..14185e0 --- /dev/null +++ b/wiki/concepts/fastapi-mongodb-role-migration.md @@ -0,0 +1,123 @@ +--- +title: "FastAPI + MongoDB — Adding Roles with Schema Validator Migration" +aliases: [fastapi-role-enum, mongodb-role-migration, fastapi-lifespan-seed, mongodb-validator-migration] +tags: [fastapi, mongodb, roles, migration, python, backend] +sources: + - "daily/2026-04-16.md" +created: 2026-04-16 +updated: 2026-04-16 +--- + +# FastAPI + MongoDB — Adding Roles with Schema Validator Migration + +Adding a new role to a FastAPI + MongoDB application touches three layers: the Python enum, the React frontend routing, and the MongoDB collection schema validator. Missing the MongoDB migration causes the application to reject user creation for the new role at the database level — silently if error handling swallows validation errors. + +## Key Points + +- **MongoDB schema validators block inserts/updates** with values not in the enum — adding a role to the Python enum alone is insufficient +- **FastAPI lifespan** is the right place for idempotent seed functions (default admin creation) — runs on every startup, safe to re-run +- When a role mirrors another role's permissions, add it alongside the existing role in **all** `require_roles()` calls and all frontend `case` statements +- Use `grep -r "REVIEWER\|reviewer"` to find every location a role is referenced before starting — avoids missed locations +- 21 occurrences was the count for the REVIEWER→add-LINGUIST change — expect 10-25 occurrences in a medium-sized monorepo + +## Details + +### The Three-Layer Change + +**Layer 1 — Python Enum (backend)** +```python +# app/models/user.py +class UserRole(str, Enum): + ADMIN = "admin" + REVIEWER = "reviewer" + LINGUIST = "linguist" # ← new +``` + +**Layer 2 — FastAPI route guards** +```python +# In every endpoint that checked for REVIEWER: +@router.get("/qc-review") +async def qc_review(user = Depends(require_roles([UserRole.REVIEWER, UserRole.LINGUIST]))): + ... +``` + +Use grep to find all `require_roles` calls: +```bash +grep -rn "require_roles\|UserRole.REVIEWER" app/ +``` + +**Layer 3 — MongoDB migration** +```python +# app/migrations/add_linguist_role.py +from app.db import get_db + +async def up(): + db = await get_db() + await db.command({ + "collMod": "users", + "validator": { + "$jsonSchema": { + "properties": { + "role": { + "enum": ["admin", "reviewer", "linguist"] # ← added + } + } + } + } + }) +``` + +Run once after deploy: `python -m app.migrations.migrator up` + +### Idempotent Seed in FastAPI Lifespan + +Seeding a default admin user in the lifespan handler ensures it exists on every fresh deploy without requiring a manual step: + +```python +# app/main.py +from contextlib import asynccontextmanager + +@asynccontextmanager +async def lifespan(app: FastAPI): + await seed_default_admin() # idempotent — no-op if admin exists + yield + +async def seed_default_admin(): + db = await get_db() + existing = await db.users.find_one({"role": "admin"}) + if not existing: + await db.users.insert_one({ + "email": "admin@example.com", + "role": "admin", + "hashed_password": hash_password("changeme"), + }) +``` + +Lifespan runs on startup — safe for production because it checks before inserting. + +### Frontend Route Mapping (React) + +When a role shares all routes with an existing role, add it to every `case` in the router/sidebar: + +```jsx +// Sidebar.jsx — route visibility +const showQCRoutes = ['reviewer', 'linguist'].includes(user.role); + +// Router switch — add alongside existing case +case 'linguist': // falls through to reviewer logic +case 'reviewer': + routes = QC_ROUTES; + break; +``` + +A `grep -r "case 'reviewer'" frontend/src/` before starting identifies all locations to change. + +## Related Concepts + +- [[wiki/tech-patterns/_index]] — FastAPI tech pattern article covers broader FastAPI setup +- [[wiki/concepts/shell-static-deploy-patterns]] — the deploy.sh fix that accompanied this role change +- [[wiki/concepts/monorepo-deploy-script-pitfall]] — the deploy script bug discovered in the same session + +## Sources + +- [[daily/2026-04-16.md]] — LINGUIST role added to Ford QC web app (FastAPI + React monorepo); 21 REVIEWER occurrences found and updated; MongoDB migration written manually; seed_default_admin added to FastAPI lifespan diff --git a/wiki/concepts/fish-abbr-patterns.md b/wiki/concepts/fish-abbr-patterns.md new file mode 100644 index 0000000..4fc6c86 --- /dev/null +++ b/wiki/concepts/fish-abbr-patterns.md @@ -0,0 +1,102 @@ +--- +title: "Fish Shell — abbr vs alias Patterns" +aliases: [fish-abbreviations, fish-abbr, fish-alias-vs-abbr] +tags: [fish, shell, dotfiles, terminal, abbr, alias] +sources: + - "daily/2026-04-17.md" +created: 2026-04-17 +updated: 2026-04-17 +--- + +# Fish Shell — abbr vs alias Patterns + +Fish shell offers two ways to create command shortcuts: `abbr` (abbreviations) and `alias`. The choice between them affects transparency and interaction with third-party Fish plugins like forgit. Using `abbr` for expanded commands and `alias` for invisible substitutions is the idiomatic Fish pattern. + +## Key Points + +- **`abbr`** expands in the command line before execution — the user sees the full command in the prompt history, making actions transparent and auditable +- **`alias`** runs as a hidden substitution — the original alias name appears in history, not the underlying command +- Use `abbr` for git/docker/uv shortcuts where seeing the expanded command is useful; use `alias` for transparent replacements like `ls→lsd`, `cat→bat` +- Fish abbreviations can conflict with forgit plugin functions — forgit defines `gco`, `grb`, `glo`, etc. as interactive functions; naming an abbreviation the same will shadow the forgit function +- Resolve forgit conflicts by renaming your abbreviation (e.g., `gco` → `gch` for `git checkout`) + +## Details + +### abbr vs alias Decision Table + +| Use case | Use | Why | +|----------|-----|-----| +| `git checkout` → `gch` | `abbr` | Expands visibly — you see `git checkout` in prompt | +| `docker compose` → `dc` | `abbr` | Expands — clear what's running | +| `ls` → `lsd` | `alias` | Should be invisible — `ls` stays in history | +| `cat` → `bat` | `alias` | Transparent replacement — looks like `cat` | +| `forgit` interactive commands | none | Provided by forgit plugin — don't shadow | + +### Defining Abbreviations + +```fish +# In ~/.config/fish/config.fish or ~/.config/fish/abbreviations.fish + +# Git shortcuts (expand visibly) +abbr --add gst 'git status' +abbr --add gch 'git checkout' # not gco — conflicts with forgit +abbr --add gpl 'git pull' +abbr --add gps 'git push' +abbr --add glo 'git log --oneline' # renamed — forgit uses glo too + +# Docker +abbr --add dc 'docker compose' +abbr --add dcu 'docker compose up -d' +abbr --add dcd 'docker compose down' + +# uv (Python package manager) +abbr --add uvr 'uv run' +abbr --add uvs 'uv sync' +``` + +### Defining Aliases (Transparent Replacements) + +```fish +# In ~/.config/fish/config.fish + +# These should feel like the original command +alias ls='lsd' +alias cat='bat --paging=never' +alias grep='rg' +alias find='fd' +``` + +### Forgit Conflict Resolution + +forgit (interactive git fzf plugin) defines these functions by default: + +| forgit function | What it does | +|----------------|--------------| +| `gco` | Interactive `git checkout` | +| `grb` | Interactive `git rebase` | +| `glo` | Interactive `git log` | +| `gbl` | Interactive `git blame` | +| `gd` | Interactive `git diff` | + +If you have abbreviations with the same names, the abbreviation wins on expansion but the forgit function is still reachable by full name. The practical fix: rename your abbreviation to avoid the conflict (`gco` → `gch`, `glo` → `glg`). + +### Viewing All Current Abbreviations + +```fish +abbr # list all abbreviations +alias # list all aliases +functions # list all functions (includes forgit) +``` + +### Persistence + +Both `abbr` and `alias` defined in `config.fish` persist across sessions. `abbr --add` called interactively creates a universal variable that also persists — but storing abbreviations in `config.fish` is more portable for dotfiles repos. + +## Related Concepts + +- [[wiki/concepts/fish-shell-path-config]] — Fish PATH configuration and SSH aliases as Fish functions +- [[wiki/dotfiles/_index]] — full dotfiles topic: Kitty, Fish, WezTerm, LazyVim, Rust CLI tools + +## Sources + +- [[daily/2026-04-17.md]] — Fish abbreviations setup session; `abbr` chosen over `alias` for git/docker/uv commands; `gco/grb/glo` conflicts with forgit resolved by renaming abbreviations; transparent replacements (`ls→lsd`, `cat→bat`) kept as `alias` diff --git a/wiki/concepts/mailcow-maildir-import.md b/wiki/concepts/mailcow-maildir-import.md new file mode 100644 index 0000000..c648ed1 --- /dev/null +++ b/wiki/concepts/mailcow-maildir-import.md @@ -0,0 +1,87 @@ +--- +title: "Mailcow — Maildir Import via rsync + doveadm" +aliases: [mailcow-import, mailcow-maildir, mailcow-doveadm, mailcow-email-import] +tags: [mailcow, email, maildir, dovecot, docker, selfhosted] +sources: + - "daily/2026-04-16.md" +created: 2026-04-16 +updated: 2026-04-16 +--- + +# Mailcow — Maildir Import via rsync + doveadm + +Importing email into Mailcow (the Docker-based mail server) requires copying Maildir files to the correct Docker volume path, fixing ownership, and running `doveadm index` — without the indexing step, mail clients will not see the imported messages even though they exist on disk. + +## Key Points + +- Mailcow stores mail in a Docker named volume: `/var/lib/docker/volumes/mailcowdockerized_vmail-vol-1/_data/{domain}/{user}/Maildir/` +- The volume is owned by the `vmail` user with **uid/gid 5000** — files must be chowned after rsync +- **`doveadm index` is mandatory** after copying — without it, Dovecot's index doesn't know about the new files and IMAP clients see nothing +- Always make a backup of the existing Maildir before importing: copy to `{user}.backup.YYYY-MM-DD` on the server +- Merge strategy: shared mailbox messages can go directly into the primary user's Inbox without subfolders + +## Details + +### Volume Path + +``` +/var/lib/docker/volumes/ + mailcowdockerized_vmail-vol-1/ + _data/ + {domain}/ e.g. ai-impress.com + {user}/ e.g. v.samoilenko + Maildir/ + cur/ delivered/read messages + new/ newly delivered messages + tmp/ temporary storage +``` + +The path is `{domain}/{user}` not `{user}@{domain}`. + +### Import Procedure + +```bash +# 1. Back up the existing mailbox +MAILDIR="/var/lib/docker/volumes/mailcowdockerized_vmail-vol-1/_data/ai-impress.com/v.samoilenko" +cp -r "$MAILDIR/Maildir" "$MAILDIR/../v.samoilenko.backup.$(date +%Y-%m-%d)" + +# 2. Copy downloaded EML files into Maildir/new/ (rsync preserves timestamps) +rsync -avz ./downloaded-mail/ "$MAILDIR/Maildir/new/" + +# 3. Fix ownership — vmail user owns the volume (uid/gid 5000) +chown -R 5000:5000 "$MAILDIR/Maildir/" + +# 4. Tell Dovecot to index the new files (runs inside the dovecot container) +docker exec mailcowdockerized-dovecot-mailcow-1 \ + doveadm index -u v.samoilenko@ai-impress.com '*' +``` + +The `'*'` glob means index all folders. For a single folder: `'INBOX'`. + +### Why doveadm index Is Required + +Dovecot maintains its own metadata index (`.dovecot.index`, `.dovecot.index.cache`) alongside the Maildir files. When files are added externally (not delivered through Dovecot's own mail delivery path), Dovecot has no record of them. IMAP clients ask Dovecot — not the filesystem — for the message list. Without indexing, `SELECT INBOX` shows an empty folder even though `cur/` contains thousands of EML files. + +### Verifying the Import + +```bash +# Count messages Dovecot can see (must match what you imported) +docker exec mailcowdockerized-dovecot-mailcow-1 \ + doveadm fetch -u v.samoilenko@ai-impress.com 'mailbox INBOX' uid | wc -l + +# Or use Mailcow webmail (SOGo) — it queries Dovecot directly +``` + +### Merging Shared Mailboxes into Inbox + +When merging multiple source mailboxes into a single user's Inbox (without creating subfolders), place all downloaded EML files into `Maildir/new/` before running `doveadm index`. Dovecot will move them to `cur/` during indexing. The merge produces a flat Inbox — no folder hierarchy. + +## Related Concepts + +- [[wiki/concepts/microsoft-graph-api-mailbox-migration]] — how to download the EML files from Microsoft 365 before this import step +- [[wiki/homelab/_index]] — Mailcow runs on self-hosted Proxmox infrastructure +- [[wiki/connections/graph-api-vs-msal-app-vs-delegated]] — auth context for the source (M365) side of the migration + +## Sources + +- [[daily/2026-04-16.md]] — ai-impress.com email migration from M365 to Mailcow; primary mailbox 2713 msgs + 8 shared mailboxes 539 msgs → 2956 msgs in single Inbox; backup created as `v.samoilenko.backup.2026-04-16` diff --git a/wiki/concepts/microsoft-graph-api-mailbox-migration.md b/wiki/concepts/microsoft-graph-api-mailbox-migration.md new file mode 100644 index 0000000..cf768b2 --- /dev/null +++ b/wiki/concepts/microsoft-graph-api-mailbox-migration.md @@ -0,0 +1,104 @@ +--- +title: "Microsoft Graph API — App-Only Mailbox Migration" +aliases: [graph-api-mail-migration, graph-api-app-only, graph-api-shared-mailboxes] +tags: [microsoft365, graph-api, email, migration, oauth, app-permissions] +sources: + - "daily/2026-04-16.md" +created: 2026-04-16 +updated: 2026-04-16 +--- + +# Microsoft Graph API — App-Only Mailbox Migration + +Migrating email from Microsoft 365 via Graph API requires an app-only (client credentials) OAuth flow, not a delegated (user) token. Delegated tokens cannot read other users' or shared mailboxes — they return 403 on any mailbox that isn't the authenticated user's own. + +## Key Points + +- **Delegated tokens cannot read shared mailboxes** — Graph API returns 403; must switch to app-only (client credentials) flow +- **`Mail.Read.All`** is the required application permission — not `Mail.Read` (delegated) +- **`User.Read.All`** application permission is only needed for `GET /users`; direct mailbox access by email (`/users/user@domain.com/mailFolders`) works without it +- Shared mailbox messages can be merged directly into another mailbox's Inbox without creating subfolders — attachments are embedded in EML +- Practical scale from 2026-04-16: primary mailbox 2713 messages / 384 MB + 8 shared mailboxes 539 messages → merged total 2956 messages / 443 MB + +## Details + +### App-Only Authentication Flow + +```python +# list_mailboxes.py — app-only token using client credentials +import msal, requests + +app = msal.ConfidentialClientApplication( + CLIENT_ID, + authority=f"https://login.microsoftonline.com/{TENANT_ID}", + client_credential=CLIENT_SECRET, +) +result = app.acquire_token_for_client( + scopes=["https://graph.microsoft.com/.default"] +) +token = result["access_token"] + +# Read a specific mailbox directly by email — no User.Read.All needed +resp = requests.get( + f"https://graph.microsoft.com/v1.0/users/{USER_EMAIL}/mailFolders", + headers={"Authorization": f"Bearer {token}"} +) +``` + +The `.default` scope acquires all application permissions configured in Azure AD — no per-call scope selection needed with client credentials. + +### Azure AD App Registration Requirements + +In Azure Portal → App Registration → API Permissions: +- Add `Mail.Read.All` as an **Application** permission (not Delegated) +- Grant admin consent — application permissions always require tenant admin consent +- `User.Read.All` application permission is only needed if you call `GET /users` to enumerate all mailboxes + +To enumerate shared mailboxes without `User.Read.All`: maintain a hardcoded list of shared mailbox emails and access each directly by email address. + +### Shared Mailbox Merge Strategy + +Shared mailbox messages merged into a primary inbox: + +```python +import requests, json + +def fetch_messages(token, mailbox_email, folder="inbox"): + url = f"https://graph.microsoft.com/v1.0/users/{mailbox_email}/mailFolders/{folder}/messages" + messages = [] + while url: + resp = requests.get(url, headers={"Authorization": f"Bearer {token}"}) + data = resp.json() + messages.extend(data.get("value", [])) + url = data.get("@odata.nextLink") # pagination + return messages + +def download_message_as_eml(token, mailbox_email, message_id): + url = f"https://graph.microsoft.com/v1.0/users/{mailbox_email}/messages/{message_id}/$value" + resp = requests.get(url, headers={"Authorization": f"Bearer {token}"}) + return resp.content # raw RFC 2822 EML bytes +``` + +The `/$value` endpoint returns the raw EML — attachments are already embedded. Save to disk and import into any IMAP server. + +### Pagination Pattern + +Graph API paginates at 10 items by default; use `$top=50` and follow `@odata.nextLink`: + +```python +url = f".../messages?$top=50" +while url: + data = requests.get(url, ...).json() + # process data["value"] + url = data.get("@odata.nextLink") +``` + +## Related Concepts + +- [[wiki/concepts/mailcow-maildir-import]] — what to do with the downloaded EML files (import into Mailcow) +- [[wiki/connections/graph-api-vs-msal-app-vs-delegated]] — choosing app-only vs delegated auth for Azure AD +- [[wiki/concepts/msal-vanilla-js-pkce]] — the user-delegated side of Azure AD OAuth + +## Sources + +- [[daily/2026-04-16.md]] — M365 mail migration for ai-impress.com → Mailcow; list_mailboxes.py rewrite from delegated to app-only token; discovered `User.Read.All` not required for direct mailbox access by email diff --git a/wiki/concepts/monorepo-deploy-script-pitfall.md b/wiki/concepts/monorepo-deploy-script-pitfall.md new file mode 100644 index 0000000..812a434 --- /dev/null +++ b/wiki/concepts/monorepo-deploy-script-pitfall.md @@ -0,0 +1,100 @@ +--- +title: "Monorepo Deploy Script — Subdirectory .git Check Pitfall" +aliases: [monorepo-git-pull, deploy-script-monorepo, git-pull-monorepo] +tags: [monorepo, git, bash, deploy, devops] +sources: + - "daily/2026-04-16.md" +created: 2026-04-16 +updated: 2026-04-16 +--- + +# Monorepo Deploy Script — Subdirectory .git Check Pitfall + +A deploy script written for a multi-repo setup (separate git repos for backend and frontend) will silently fail to pull new code when the project is converted to a monorepo. The script checks for `.git` in subdirectories that don't exist in a monorepo, never finds them, skips the `git pull`, and exits successfully — leaving the server running old code with no warning. + +## Key Points + +- **Silent failure**: the script exits 0 even though `git pull` never ran — nothing in stdout indicates the problem +- The symptom: running `scripts/deploy.sh` reports success, but the server is still running the old version +- **Fix**: in a monorepo, `git pull` runs once at the repo root — remove all subdirectory `.git` checks +- The warning message (if the script emitted one) is easy to miss in CI/CD output or when tailing logs +- Multi-repo scripts often check `backend/.git` and `frontend/.git` to determine if each component needs to be pulled separately + +## Details + +### The Anti-Pattern + +```bash +# scripts/deploy.sh — written for multi-repo, broken in monorepo + +if [ -d "backend/.git" ]; then + echo "Pulling backend..." + cd backend && git pull && cd .. +fi + +if [ -d "frontend/.git" ]; then + echo "Pulling frontend..." + cd frontend && git pull && cd .. +fi + +# In a monorepo: neither condition is true → git pull never runs +# Script exits 0 anyway → deploy "succeeds" with stale code +``` + +### The Fix + +```bash +# scripts/deploy.sh — monorepo version + +set -e +ROOT_DIR="$(cd "$(dirname "$0")/.." && pwd)" +cd "$ROOT_DIR" + +echo "Pulling latest code..." +git pull + +echo "Rebuilding backend..." +cd backend && pip install -r requirements.txt && cd .. + +echo "Rebuilding frontend..." +cd frontend && npm ci && npm run build && cd .. + +echo "Restarting services..." +systemctl restart myapp-backend +systemctl reload nginx +``` + +One `git pull` at the root covers all subdirectories in a monorepo. + +### Detecting the Problem + +If a deploy claims success but the running version doesn't match the expected commit: + +```bash +# On the server — check what commit is actually running +git log -1 --oneline + +# Compare with what should be deployed +git fetch origin main +git log -1 --oneline origin/main +``` + +If they differ after a "successful" deploy, the deploy script didn't actually pull. + +### Coexisting Deploy Scripts + +Repositories that transitioned from multi-repo to monorepo sometimes have two deploy scripts: +- `full-deploy.sh` — the original multi-repo script (potentially still working for other components) +- `scripts/deploy.sh` — the active script that was incorrectly carried over + +Always identify which script is actually invoked by the CI/CD system or cron job before modifying. + +## Related Concepts + +- [[wiki/concepts/shell-static-deploy-patterns]] — broader deploy script patterns including cp pitfalls and web server reload +- [[wiki/concepts/fastapi-mongodb-role-migration]] — the feature deployed in the same session where this bug was found +- [[wiki/architecture/_index]] — Docker Compose deployment patterns used alongside these scripts + +## Sources + +- [[daily/2026-04-16.md]] — `scripts/deploy.sh` in Ford QC web app discovered checking `backend/.git` and `frontend/.git`; monorepo never matched; fixed to `git pull` at root diff --git a/wiki/concepts/nextjs-basepath-auth-redirects.md b/wiki/concepts/nextjs-basepath-auth-redirects.md new file mode 100644 index 0000000..25a352d --- /dev/null +++ b/wiki/concepts/nextjs-basepath-auth-redirects.md @@ -0,0 +1,110 @@ +--- +title: "Next.js basePath — Auth Redirect Gotcha" +aliases: [nextjs-basepath, basepath-redirects, nextjs-basepath-auth] +tags: [nextjs, basepath, auth, msal, redirect, deployment] +sources: + - "daily/2026-04-16.md" +created: 2026-04-16 +updated: 2026-04-16 +--- + +# Next.js basePath — Auth Redirect Gotcha + +When a Next.js app is deployed under a sub-path (e.g., `/hp-prod-tracker`), all auth redirect URLs must include the `basePath` prefix. Hardcoded paths like `/login` or `/pending` silently resolve to the root domain — the redirect succeeds from Next.js's perspective but the browser navigates to the wrong URL, breaking the auth flow. + +## Key Points + +- **All redirect paths must include basePath**: `/hp-prod-tracker/login`, not `/login` +- Next.js's `router.push()` automatically prepends `basePath`, but **string redirects in middleware or API routes do not** — these require manual prefixing +- `Response.redirect()` and `NextResponse.redirect()` require the full path including basePath +- Sign-out redirects and pending-state redirects are the most common locations where this is missed +- The bug is environment-specific: works in local dev (where basePath is not active or the app is at root) but breaks in production + +## Details + +### Configuring basePath in Next.js + +```js +// next.config.js +module.exports = { + basePath: '/hp-prod-tracker', +} +``` + +With this set, `router.push('/login')` navigates to `/hp-prod-tracker/login`. But manual redirects bypass this: + +```ts +// ❌ WRONG — in middleware.ts or API route +return NextResponse.redirect(new URL('/login', request.url)); +// Redirects to https://domain.com/login (missing basePath) + +// ✅ CORRECT +const basePath = '/hp-prod-tracker'; +return NextResponse.redirect(new URL(`${basePath}/login`, request.url)); +``` + +### Auth Flow Locations That Require Manual Fixing + +1. **Middleware auth guard** — redirect to login if no session: + ```ts + // middleware.ts + const BASE = process.env.NEXT_PUBLIC_BASE_PATH || ''; + return NextResponse.redirect(new URL(`${BASE}/login`, request.url)); + ``` + +2. **Sign-out handler** — after clearing session: + ```ts + // api/auth/signout route + return NextResponse.redirect(new URL(`${BASE}/login`, request.url)); + ``` + +3. **Pending page redirect** — when org domain doesn't match: + ```ts + // If user.org domain not found → pending + return NextResponse.redirect(new URL(`${BASE}/pending`, request.url)); + ``` + +4. **MSAL redirect URI** — must match the Azure Portal registration exactly: + ```ts + const msalConfig = { + auth: { + redirectUri: window.location.origin + '/hp-prod-tracker/login.html' + } + }; + ``` + +### Environment Variable Strategy + +Use `NEXT_PUBLIC_BASE_PATH` (exposed to the browser) alongside `basePath` in next.config.js: + +```bash +# .env.production +NEXT_PUBLIC_BASE_PATH=/hp-prod-tracker +``` + +```ts +// Shared utility +export const BASE_PATH = process.env.NEXT_PUBLIC_BASE_PATH ?? ''; +export const redirect = (path: string) => `${BASE_PATH}${path}`; +``` + +This makes the basePath a single source of truth across middleware, API routes, and client code. + +### Organization Domain Matching (SSO Context) + +After successful Azure AD SSO, the application checks `organizations.domain` in the database to determine if the user belongs to an active organization. If no matching domain is found, the user is redirected to `/pending`. Ensure the `organizations` table has a row with `domain = 'oliver.agency'` (or whichever email domain the users log in with). + +```sql +-- After first SSO deploy, verify this exists: +SELECT * FROM organizations WHERE domain = 'oliver.agency'; +``` + +## Related Concepts + +- [[wiki/concepts/msal-vanilla-js-pkce]] — MSAL.js client-side auth implementation details +- [[wiki/tech-patterns/azure-ad-msal-auth]] — Azure AD app registration and redirect URI configuration +- [[wiki/connections/graph-api-vs-msal-app-vs-delegated]] — auth flow selection context + +## Sources + +- [[daily/2026-04-16.md]] — HP Prod Tracker SSO migration (OAuthRelay → MSAL.js client-side); basePath issue discovered post-deploy where sign-out and pending redirects went to root domain instead of `/hp-prod-tracker/*`; fixed by prefixing all redirect strings with basePath diff --git a/wiki/concepts/openai-max-completion-tokens.md b/wiki/concepts/openai-max-completion-tokens.md new file mode 100644 index 0000000..2c19c52 --- /dev/null +++ b/wiki/concepts/openai-max-completion-tokens.md @@ -0,0 +1,110 @@ +--- +title: "OpenAI API — max_completion_tokens Migration" +aliases: [openai-max-tokens, max_completion_tokens, openai-param-migration] +tags: [openai, api, llm, python, migration, wrapper] +sources: + - "daily/2026-04-17.md" +created: 2026-04-17 +updated: 2026-04-17 +--- + +# OpenAI API — max_completion_tokens Migration + +Newer OpenAI models (o-series, GPT-4.1, GPT-5.x) no longer accept `max_tokens` in the API request. The parameter was renamed to `max_completion_tokens`. Passing `max_tokens` causes an explicit API error that surfaces in two places: the raw API call site and any internal wrapper functions that forward the parameter. + +## Key Points + +- `max_tokens` is **rejected** by newer OpenAI models — the API returns an error, not a warning +- Error message: `'max_tokens' is not supported with this model` +- The fix must be applied in **both** direct API calls and any internal wrapper that passes `max_tokens` as a keyword argument (e.g., a `chat_structured()` helper) +- A frontend crash (e.g., `TypeError: Cannot read properties of undefined`) downstream of this error is often a symptom — the backend returned an error instead of the expected response, and the frontend tried to process `undefined` +- Always grep for `max_tokens` in both the API call layer and any utility/wrapper modules before deploying to a project using newer models + +## Details + +### The Error + +``` +openai.BadRequestError: 400 Bad Request +{'error': {'message': "'max_tokens' is not supported with this model", 'type': 'invalid_request_error', ...}} +``` + +This is a hard error — the API call fails, returns no completion, and the caller receives `None` or raises an exception depending on the error handling. + +### Fix: Direct API Call + +```python +# ❌ Old — fails on newer models +response = client.chat.completions.create( + model="gpt-4.1", + messages=messages, + max_tokens=2048, +) + +# ✅ New +response = client.chat.completions.create( + model="gpt-4.1", + messages=messages, + max_completion_tokens=2048, +) +``` + +### Fix: Internal Wrappers + +Any wrapper function that accepts and forwards `max_tokens` must also be updated: + +```python +# ❌ Old wrapper +def chat_structured(messages, schema, max_tokens=2048): + return client.chat.completions.create( + ..., + max_tokens=max_tokens, # fails + ) + +# ✅ New wrapper +def chat_structured(messages, schema, max_completion_tokens=2048): + return client.chat.completions.create( + ..., + max_completion_tokens=max_completion_tokens, + ) +``` + +If the wrapper is used across multiple call sites, update the parameter name in callers too — or keep the old kwarg name in the function signature but map it internally for backwards compatibility. + +### Finding All Affected Call Sites + +```bash +# Find every location that passes max_tokens +grep -rn "max_tokens" app/ --include="*.py" + +# Common locations: +# - Direct client.chat.completions.create() calls +# - Wrapper functions like chat_structured(), llm_call(), generate_text() +# - Any function that accepts **kwargs and forwards to OpenAI +``` + +### Frontend Symptom + +If the backend API returns an error instead of the expected JSON, frontend JavaScript that tries to process the response will crash with a TypeError on `undefined`: + +``` +TypeError: Cannot read properties of undefined (reading 'split') +``` + +This appears in minified bundles as a stack trace pointing to the component that processes the LLM response (e.g., `VariantsGrid.tsx`). Fixing the backend error resolves the frontend crash — the frontend null-guard is optional hardening, not the root fix. + +### Model Compatibility + +- Models that still accept `max_tokens`: GPT-3.5-turbo, GPT-4 (original), some fine-tuned variants +- Models that require `max_completion_tokens`: o1, o3, GPT-4.1, GPT-5.x and later +- When in doubt, use `max_completion_tokens` — older models that don't recognize it may silently ignore it, which is preferable to a hard error + +## Related Concepts + +- [[wiki/concepts/fastapi-mongodb-role-migration]] — FastAPI backend patterns where LLM wrappers live +- [[wiki/llm-models/_index]] — model catalog for identifying which models require the new parameter +- [[wiki/tech-patterns/_index]] — AI tech pattern articles covering OpenAI integration + +## Sources + +- [[daily/2026-04-17.md]] — Barclays banner builder app on optical-dev.oliver.solutions; `max_tokens` error surfaced during Barclays Rainy Day Saver brief testing; fix applied to both direct API call and `chat_structured()` wrapper; deployed with `bash deploy.sh --skip-frontend` diff --git a/wiki/concepts/proxmox-mcp-server.md b/wiki/concepts/proxmox-mcp-server.md new file mode 100644 index 0000000..3c88f17 --- /dev/null +++ b/wiki/concepts/proxmox-mcp-server.md @@ -0,0 +1,80 @@ +--- +title: "Proxmox MCP Server — Claude Code Integration" +aliases: [proxmox-mcp, canvrno-proxmox-mcp, mcp-proxmox] +tags: [proxmox, mcp, claude-code, homelab, vm-management] +sources: + - "daily/2026-04-18.md" +created: 2026-04-18 +updated: 2026-04-18 +--- + +# Proxmox MCP Server — Claude Code Integration + +There is no official Proxmox MCP server, but several community implementations exist. The most widely adopted is `canvrno/ProxmoxMCP` — a Python-based server that allows natural-language VM management directly from Claude Code. Installing it to the global Claude Code config makes it available in every project session. + +## Key Points + +- **No official Proxmox MCP** — all available servers are community-built +- **`canvrno/ProxmoxMCP`** is the recommended choice: Python, 175+ GitHub stars, active maintenance, good documentation (as of 2026-04) +- **`mcp-server-proxmox`** (EricWvi) is a TypeScript alternative — less popular +- **Proxmox NLP Server** (Ik0ri4n) works via SSH rather than the Proxmox API directly +- Install to global Claude Code config (`~/.claude/settings.json`) so it's available from any project directory — same pattern as GitHub and Bitbucket MCP servers + +## Details + +### Available Proxmox MCP Servers + +| Server | Language | Approach | Stars (2026-04) | +|--------|----------|----------|-----------------| +| `canvrno/ProxmoxMCP` | Python | Proxmox REST API | 175+ | +| `EricWvi/mcp-server-proxmox` | TypeScript | Proxmox REST API | Lower | +| `Ik0ri4n/proxmox-nlp-server` | — | SSH + pve commands | Niche | + +`canvrno/ProxmoxMCP` is the practical choice: Python aligns with the rest of the homelab tooling, and the REST API approach is more robust than SSH command strings. + +### Global Config Installation + +Add to `~/.claude/settings.json` alongside other MCP servers: + +```json +{ + "mcpServers": { + "proxmox": { + "command": "uvx", + "args": ["proxmox-mcp"], + "env": { + "PROXMOX_HOST": "192.168.x.x", + "PROXMOX_USER": "root@pam", + "PROXMOX_PASSWORD": "your-password", + "PROXMOX_NODE": "pve" + } + } + } +} +``` + +After editing, restart Claude Code for the server to connect. The MCP tools appear under the `proxmox__*` namespace. + +### What It Enables + +With the Proxmox MCP installed, Claude Code can: +- Create, start, stop, and delete VMs by natural language +- Query VM status and resource usage +- Manage containers (LXC) +- Execute tasks that would otherwise require SSH into the Proxmox host or the web UI + +This complements but does not replace the Proxmox web UI — complex storage and network configuration is still easier in the UI. + +### Kali Linux Context (2026-04-18) + +The Proxmox MCP was installed in the context of setting up a Kali Linux VM for authorized penetration testing of the user's own infrastructure. The Kali VM was created manually via SSH + `qm` commands before the MCP was added; future VMs can be created directly from Claude Code. + +## Related Concepts + +- [[wiki/concepts/proxmox-vm-management-methods]] — all approaches for Proxmox VM management (API, Terraform, Ansible, SSH+qm) +- [[wiki/concepts/bitbucket-mcp-atlassian]] — same global config installation pattern for Bitbucket and GitHub MCP servers +- [[wiki/homelab/_index]] — Proxmox install, IOMMU, hypervisor setup articles + +## Sources + +- [[daily/2026-04-18.md]] — User asked about Proxmox MCP; `canvrno/ProxmoxMCP` selected over TypeScript and SSH alternatives; added to global Claude Code config; Kali VM context for authorized server pentesting diff --git a/wiki/concepts/proxmox-vm-management-methods.md b/wiki/concepts/proxmox-vm-management-methods.md new file mode 100644 index 0000000..edb8982 --- /dev/null +++ b/wiki/concepts/proxmox-vm-management-methods.md @@ -0,0 +1,112 @@ +--- +title: "Proxmox VM Management Approaches" +aliases: [proxmox-vm-creation, proxmox-api, proxmox-terraform, qm-commands] +tags: [proxmox, vm, infrastructure, homelab, iac, terraform, ansible] +sources: + - "daily/2026-04-18.md" +created: 2026-04-18 +updated: 2026-04-18 +--- + +# Proxmox VM Management Approaches + +Proxmox VE exposes several interfaces for creating and managing VMs. The right choice depends on whether the task is one-off or repeatable, and how much automation is required. SSH + `qm` commands is the fastest path for ad-hoc work; the Proxmox REST API is the best foundation for repeatable automation. + +## Key Points + +- **Proxmox REST API** — the canonical approach; can be called via `curl` or the Python `proxmoxer` SDK without SSH +- **Terraform + Proxmox provider** (`bpg/terraform-provider-proxmox`) — IaC approach; best for repeatable, version-controlled infrastructure +- **Ansible** — playbook-based automation; good when the same VM config is deployed multiple times or when post-provision config is also needed +- **SSH + `qm` commands** — fastest for one-off tasks; `qm create`, `qm start`, `qm stop`, `qm destroy`; no additional tooling needed +- **Proxmox MCP** — Claude Code natural-language interface wrapping the REST API; good for interactive, conversational VM management + +## Details + +### Comparison Table + +| Approach | Setup Cost | Repeatable | Version-Controlled | Interactive | +|----------|-----------|------------|-------------------|-------------| +| SSH + qm | None | Low | No | Yes | +| Proxmox REST API | Low | High | With scripts | No | +| Terraform | Medium | High | Yes (tfstate) | No | +| Ansible | Medium | High | Yes | No | +| Proxmox MCP | Low (one-time config) | Medium | No | Yes | + +### SSH + qm Commands + +The built-in CLI on the Proxmox host. No extra software needed — just SSH access: + +```bash +# Create a VM +qm create 200 --name kali-linux --memory 4096 --cores 4 --net0 virtio,bridge=vmbr0 + +# Attach ISO +qm set 200 --ide2 local:iso/kali-linux-2024.2-installer-amd64.iso,media=cdrom +qm set 200 --scsi0 local-lvm:32 # 32 GB disk +qm set 200 --boot order=ide2 + +# Lifecycle +qm start 200 +qm status 200 +qm stop 200 +qm destroy 200 +``` + +Best for: ad-hoc VM creation, quick testing, situations where Terraform/Ansible is overkill. + +### Proxmox REST API (Python proxmoxer) + +```python +from proxmoxer import ProxmoxAPI + +proxmox = ProxmoxAPI('192.168.x.x', user='root@pam', password='...', verify_ssl=False) + +# Create VM via API +proxmox.nodes('pve').qemu.post( + vmid=201, + name='new-vm', + memory=4096, + cores=4, + net0='virtio,bridge=vmbr0', +) +proxmox.nodes('pve').qemu(201).status.start.post() +``` + +Best for: scripted automation without IaC overhead, integrations with other systems. + +### Terraform (bpg/terraform-provider-proxmox) + +```hcl +resource "proxmox_virtual_environment_vm" "kali" { + name = "kali-linux" + node_name = "pve" + + cpu { cores = 4 } + memory { dedicated = 4096 } + + disk { + datastore_id = "local-lvm" + size = 32 + interface = "scsi0" + } +} +``` + +Best for: production infrastructure, reproducible environments, teams needing code review for VM changes. + +### Choosing an Approach + +- **One-off / exploration:** SSH + qm or Proxmox MCP +- **Scripted repeatable (no IaC):** Proxmox REST API (proxmoxer) +- **IaC / team environment:** Terraform +- **Post-provision config included:** Ansible (can call Proxmox API + configure the VM) +- **Claude Code interactive session:** Proxmox MCP + +## Related Concepts + +- [[wiki/concepts/proxmox-mcp-server]] — MCP server that wraps the Proxmox REST API for Claude Code use +- [[wiki/homelab/_index]] — Proxmox installation, IOMMU, hypervisor configuration + +## Sources + +- [[daily/2026-04-18.md]] — User asked if SSH+qm is the only way to create VMs from Claude Code; assistant outlined all four approaches (Proxmox API, Terraform, Ansible, SSH+qm); discussion led to discovering Proxmox MCP as a fifth option diff --git a/wiki/concepts/python-service-deployment-dotenv.md b/wiki/concepts/python-service-deployment-dotenv.md new file mode 100644 index 0000000..4e599d0 --- /dev/null +++ b/wiki/concepts/python-service-deployment-dotenv.md @@ -0,0 +1,107 @@ +--- +title: "Python Service Deployment — venv and .env Checklist" +aliases: [python-deploy-checklist, python-venv-deploy, dotenv-deployment, service-deploy-python] +tags: [python, deployment, venv, dotenv, bash, linux, systemd] +sources: + - "daily/2026-04-16.md" +created: 2026-04-16 +updated: 2026-04-16 +--- + +# Python Service Deployment — venv and .env Checklist + +Deploying a new version of a Python service that adds dependencies or switches to `.env`-based configuration requires manual steps on the server that are easily missed. The service may fail to start silently (if the systemd unit is already running), or start with `ModuleNotFoundError` or `ConfigurationError` that only appears in `journalctl`. + +## Key Points + +- After `git pull`, **always check if new packages were added to requirements.txt** — the venv on the server doesn't update automatically +- Install into the **correct venv**: use `/path/to/venv/bin/pip install`, not the system pip +- When a service transitions from hardcoded config to `utils/config.py + .env`, the `.env` file must be created on every server — the service won't start without it +- **`.env.example` with sensible defaults** is sufficient for production when actual secrets come from other mechanisms (e.g., Box config JSON, environment-specific config files) +- Multi-environment servers (dev + prod side by side) require the checklist run once per environment + +## Details + +### Deployment Checklist for Python Service Updates + +```bash +# 1. Pull the latest code +cd /home/box-cli/FORD_SCRIPTS/ford_qc_git_dev/ford_qc +git pull + +# 2. Check for new dependencies +diff requirements.txt <(path/to/venv/bin/pip freeze) +# Or just reinstall — pip is idempotent for existing packages +/path/to/venv/bin/pip install -r requirements.txt + +# 3. Create .env if it doesn't exist +if [ ! -f .env ]; then + cp .env.example .env + echo "Created .env from .env.example — review before production" +fi + +# 4. Restart the service +sudo systemctl restart ford-qc-hotfolder + +# 5. Verify it's running +sudo systemctl status ford-qc-hotfolder +sudo journalctl -u ford-qc-hotfolder -n 50 --no-pager +``` + +### Finding the Correct venv Path + +```bash +# The venv is typically next to the service code +ls /path/to/service/ +# Look for: venv/, .venv/, env/, or a virtualenv/ directory + +# Confirm it's the right one +/path/to/venv/bin/python --version +/path/to/venv/bin/pip list | grep python-dotenv +``` + +### Common Failure Sequence + +1. `git pull` succeeds +2. `systemctl restart` succeeds (systemd starts the new process) +3. Process crashes immediately — systemd restarts it a few times then gives up +4. `systemctl status` shows `Active: failed` +5. `journalctl -u service-name -n 20` reveals the actual error: + - `ModuleNotFoundError: No module named 'dotenv'` → pip install missing + - `ConfigurationError: Missing required configuration variables` → .env not created + +### .env.example Strategy + +Keep `.env.example` committed to the repo with all variables and reasonable defaults. Actual secrets go in `.env` (gitignored). For services where secrets come from a separate mechanism: + +```ini +# .env.example — safe to commit, reasonable defaults for most deployments +LOG_LEVEL=INFO +BOX_CONFIG_PATH=/etc/service/box_config.json +POLL_INTERVAL_SECONDS=30 +MAX_RETRIES=3 +``` + +The `.env.example` approach means `cp .env.example .env` is always a valid starting point — no manual env var hunting needed on new servers. + +### Multi-Environment Server Layout (Ford QC Pattern) + +``` +/home/box-cli/FORD_SCRIPTS/ +├── ford_qc_git_dev/ +│ └── ford_qc/ # dev environment — deploy and test here first +└── ford_qc_git_prod/ + └── ford_qc/ # prod — apply after team approval +``` + +Both environments need the same checklist run independently. Dev first, prod only after team sign-off. + +## Related Concepts + +- [[wiki/concepts/monorepo-deploy-script-pitfall]] — another deploy script failure mode found in the same codebase +- [[wiki/concepts/shell-static-deploy-patterns]] — shell deploy patterns for static frontends (complementary) +- [[wiki/client-knowledge/_index]] — Ford project context including ford-qc-hotfolder service + +## Sources + +- [[daily/2026-04-16.md]] — Ford QC hotfolder service failed on `box-cli-01` after git pull; `ModuleNotFoundError: No module named 'dotenv'` (new dependency); then `ConfigurationError` (new `.env` requirement from config.py refactor); resolved with pip install + cp .env.example .env diff --git a/wiki/connections/_index.md b/wiki/connections/_index.md index 0f913a7..d6157f0 100644 --- a/wiki/connections/_index.md +++ b/wiki/connections/_index.md @@ -6,6 +6,7 @@ | Article | Connects | Source | Updated | |---------|---------|--------|---------| | [[wiki/connections/oauth-state-mismatch-debugging]] | LibreChat OpenID ↔ MSAL SPA ↔ Azure AD — state mismatch root cause shared across implementations | daily/2026-04-15.md | 2026-04-15 | +| [[wiki/connections/graph-api-vs-msal-app-vs-delegated]] | Graph API app-only ↔ MSAL delegated — choosing the right Azure AD auth flow; why delegated 403s on shared mailboxes | daily/2026-04-16.md | 2026-04-16 | diff --git a/wiki/connections/graph-api-vs-msal-app-vs-delegated.md b/wiki/connections/graph-api-vs-msal-app-vs-delegated.md new file mode 100644 index 0000000..0804090 --- /dev/null +++ b/wiki/connections/graph-api-vs-msal-app-vs-delegated.md @@ -0,0 +1,58 @@ +--- +title: "Connection: App-Only vs Delegated Auth — Graph API and MSAL" +connects: + - "concepts/microsoft-graph-api-mailbox-migration" + - "concepts/msal-vanilla-js-pkce" + - "tech-patterns/azure-ad-msal-auth" +sources: + - "daily/2026-04-16.md" +created: 2026-04-16 +updated: 2026-04-16 +--- + +# Connection: App-Only vs Delegated Auth — Graph API and MSAL + +## The Connection + +Microsoft Azure AD supports two fundamentally different OAuth flows — app-only (client credentials) and delegated (user) — and the correct choice depends entirely on whether a human user is present in the flow. MSAL.js in a browser SPA always uses delegated auth (the user logs in). Graph API scripts for admin tasks like mailbox migration must use app-only auth. Using the wrong flow causes a 403 that appears to be a permissions problem but is actually an authentication architecture problem. + +## Key Insight + +**Delegated tokens are bounded by what the user themselves can do.** A delegated token for user A cannot read user B's mailbox, even if the app has `Mail.Read.All` *delegated* permission — because user A doesn't have access to user B's mailbox. Delegated `Mail.Read.All` means "read mail for any user this specific user is authorized to impersonate" — which is typically nobody else. + +**App-only tokens represent the application itself**, not any user. With `Mail.Read.All` as an *application* permission (admin-consented), the app can read any mailbox in the tenant. This is the correct architecture for server-to-server operations: email migration, backup, monitoring. + +The error from using the wrong flow: `403 Forbidden` with body `{"code": "ErrorAccessDenied", "message": "Access is denied."}`. This looks like a permissions error, leading developers to add more permissions — which doesn't help because the problem is the flow type, not the permission name. + +## When to Use Each Flow + +| Scenario | Flow | Why | +|----------|------|-----| +| User logs into web app | Delegated (MSAL.js PKCE) | Token is for the user's own data | +| Script reads all mailboxes | App-only (client credentials) | No user present; needs cross-tenant access | +| Server validates user's token | App-only for JWKS validation | Server-side, no user interaction | +| User approves access to their own email | Delegated | User consents explicitly | +| Bulk email export/migration | App-only | Admin task, no interactive user | +| LibreChat / SSO login | Delegated (OpenID) | User must authenticate themselves | + +## Evidence + +- **Delegated fails on shared mailboxes (2026-04-16):** `list_mailboxes.py` initially used a delegated token. Worked for the primary mailbox owner but returned 403 for all 8 shared mailboxes. Rewriting to app-only with `Mail.Read.All` application permission resolved it immediately. +- **MSAL.js SPA (2026-04-15):** `social-reporting-tool` and HP Prod Tracker both use MSAL.js delegated flow — correct because a human user is authenticating and the app only reads that user's data. +- **JWKS validation (both):** The backend validates tokens using the JWKS endpoint — this is not "delegated" or "app-only", it's offline validation using Azure's public keys. It works for tokens from either flow. + +## Diagnostic Heuristic + +When you see 403 from Graph API: + +1. Is there a human user in the flow right now? → Use delegated +2. Is this a server script or background job? → Use app-only +3. Did you check **application** permissions vs **delegated** permissions in Azure Portal? → They have the same names but are different grant types +4. Did you grant admin consent for application permissions? → Required; delegated permissions sometimes work without admin consent + +## Related Concepts + +- [[wiki/concepts/microsoft-graph-api-mailbox-migration]] — app-only implementation for M365 mailbox migration +- [[wiki/concepts/msal-vanilla-js-pkce]] — delegated flow implementation for SPAs +- [[wiki/concepts/librechat-openid-auth]] — delegated OpenID flow in LibreChat +- [[wiki/tech-patterns/azure-ad-msal-auth]] — Azure AD patterns across Oliver Agency projects diff --git a/wiki/dotfiles/wezterm-config.md b/wiki/dotfiles/wezterm-config.md index 9ddc41c..f8df9af 100644 --- a/wiki/dotfiles/wezterm-config.md +++ b/wiki/dotfiles/wezterm-config.md @@ -2,9 +2,11 @@ title: "WezTerm Configuration Guide" aliases: [wezterm-lua-config, wezterm-setup, wezterm-config-files] tags: [wezterm, terminal, lua, dotfiles, config] -sources: [raw/Configuration - Wez's Terminal Emulator.md] +sources: + - "raw/Configuration - Wez's Terminal Emulator.md" + - "daily/2026-04-16.md" created: 2026-04-17 -updated: 2026-04-17 +updated: 2026-04-16 --- ## Overview @@ -100,6 +102,45 @@ Convention: modules export an `apply_to_config(config)` function that mutates th - [[wiki/dotfiles/terminal-cheatsheet|Terminal Cheatsheet]] — daily WezTerm + Fish stack reference - [[wiki/dotfiles/linux-terminal-ricing|Linux Terminal Ricing]] — full setup guide with themes +## Personal Setup (macOS — 2026-04-16) + +Actual config applied to `~/.wezterm.lua` matching Ghostty/Kitty preferences: + +```lua +local wezterm = require 'wezterm' +local config = wezterm.config_builder() + +-- Appearance (matches Ghostty/Kitty stack) +config.color_scheme = 'Catppuccin Mocha' +config.font = wezterm.font('FiraCode Nerd Font Mono', { weight = 'Regular' }) +config.font_size = 16 +config.harfbuzz_features = { 'calt', 'clig', 'liga', 'ss01', 'ss02' } -- FiraCode ligatures +config.window_background_opacity = 0.95 +config.macos_window_background_blur = 20 + +-- Performance (Apple Silicon / ProMotion) +config.front_end = 'WebGpu' -- requires full restart to take effect +config.max_fps = 120 -- ProMotion display +config.animation_fps = 120 + +-- Shell +config.default_prog = { '/opt/homebrew/bin/fish', '-l' } + +-- Leader key (tmux-style) +config.leader = { key = 'a', mods = 'CTRL', timeout_milliseconds = 1000 } + +-- Scrollback +config.scrollback_lines = 50000 + +return config +``` + +Notes: +- `front_end = 'WebGpu'` (Metal on macOS) — live reload doesn't apply; requires full WezTerm restart +- FiraCode ligatures via `harfbuzz_features` — not enabled by default +- Leader key `⌃A` chosen for tmux familiarity; 1 second timeout before it acts as literal Ctrl+A + ## Sources - `raw/Configuration - Wez's Terminal Emulator.md` (clipped from wezterm.org/config/files.html) +- [[daily/2026-04-16.md]] — personal WezTerm config created matching Catppuccin Mocha stack; extended with WebGpu renderer, leader key, workspace, 50k scrollback, status bar diff --git a/wiki/log.md b/wiki/log.md index bbd138f..7ee68cd 100644 --- a/wiki/log.md +++ b/wiki/log.md @@ -13,6 +13,21 @@ - All three hooks promoted to global (fire from any project directory) - Compile time set to 21:00 (9 PM) BST +## [2026-04-18T22:10:00+01:00] compile | 2026-04-17.md +- Source: daily/2026-04-17.md +- Articles created: [[wiki/concepts/openai-max-completion-tokens]], [[wiki/concepts/fish-abbr-patterns]], [[wiki/concepts/bitbucket-mcp-atlassian]] +- Articles updated: (none) + +## [2026-04-18T23:00:00+01:00] compile | 2026-04-18.md +- Source: daily/2026-04-18.md +- Articles created: [[wiki/concepts/proxmox-mcp-server]], [[wiki/concepts/proxmox-vm-management-methods]] +- Articles updated: (none) + +## [2026-04-18T21:53:15+01:00] compile | 2026-04-16.md +- Source: daily/2026-04-16.md +- Articles created: [[wiki/concepts/microsoft-graph-api-mailbox-migration]], [[wiki/concepts/mailcow-maildir-import]], [[wiki/concepts/fastapi-mongodb-role-migration]], [[wiki/concepts/monorepo-deploy-script-pitfall]], [[wiki/concepts/nextjs-basepath-auth-redirects]], [[wiki/concepts/python-service-deployment-dotenv]], [[wiki/connections/graph-api-vs-msal-app-vs-delegated]] +- Articles updated: [[wiki/dotfiles/wezterm-config]] (added Personal Setup section — Catppuccin Mocha, FiraCode, WebGpu, leader key) + ## [2026-04-17T16:03:42+01:00] compile | 2026-04-15.md - Source: daily/2026-04-15.md - Articles created: [[wiki/concepts/librechat-openid-auth]], [[wiki/concepts/msal-vanilla-js-pkce]], [[wiki/concepts/shell-static-deploy-patterns]], [[wiki/concepts/fish-shell-path-config]], [[wiki/connections/oauth-state-mismatch-debugging]]