diff --git a/99 Daily/2026-04-24.md b/99 Daily/2026-04-24.md index 443c99d..2245da1 100644 --- a/99 Daily/2026-04-24.md +++ b/99 Daily/2026-04-24.md @@ -86,3 +86,21 @@ tags: [daily] - 11:06 (1min) | `memory-compiler` - **Asked:** Create a schema for compiling daily conversation logs into structured wiki articles. - **Done:** Set up AGENTS.md schema with compiler analogy, defined daily/ directory structure, and created knowledge extraction pattern. +- 11:07 (1min) | `ai_leed` + - **Asked:** Check that Claude, Obsidian, and cc-dashboard integrations and hooks are configured correctly for this machine. + - **Done:** Verified all three path modes work correctly and added dynamic topic detection in session-start.py hook. +- 11:11 | `memory-compiler` + - **Asked:** How should conversation logs be compiled into structured wiki articles? + - **Done:** Defined a schema for converting daily conversation logs into organized concept articles with key insights. +- 11:12 (<1min) | `memory-compiler` + - **Asked:** How should a knowledge compiler extract and structure insights from daily conversation logs? + - **Done:** Documented a compiler schema for converting conversation logs into structured wiki articles using a source-code analogy. +- 11:15 (2min) | `memory-compiler` + - **Asked:** Build a knowledge compiler system to extract structured wiki articles from daily conversation logs. + - **Done:** Created AGENTS.md schema documenting the compiler architecture and compiled 2 concept articles from a sample conversation log. +- 11:17 (2min) | `ai_leed` + - **Asked:** Organized task list by sections and created Obsidian work plan for today. + - **Done:** Verified all Claude, Obsidian, and CC-Dashboard integrations and hooks are correctly configured for this machine. +- 11:18 | `memory-compiler` + - **Asked:** Create a knowledge compiler system that extracts structured wiki articles from daily conversation logs. + - **Done:** Designed AGENTS.md schema adapting Karpathy's LLM knowledge base to compile personal AI conversation knowledge into wiki articles. diff --git a/wiki/_master-index.md b/wiki/_master-index.md index 109be9c..e1e0340 100644 --- a/wiki/_master-index.md +++ b/wiki/_master-index.md @@ -23,7 +23,7 @@ This 3-hop pattern works for hundreds of articles without vector search. | [[wiki/tech-patterns/_index\|tech-patterns/]] | Recurring tech stacks: FastAPI, React/Vite, Next.js, Azure AD, AI, Box, One2Edit | 9 | | [[wiki/architecture/_index\|architecture/]] | Cross-cutting architectural patterns: Docker Compose, multi-agent AI, GCP timeout, RAG, hotfolder | 5 | | [[wiki/client-knowledge/_index\|client-knowledge/]] | Per-client notes for Ford, H&M, L'Oréal (2+ projects each) | 3 | -| [[wiki/concepts/_index\|concepts/]] | Atomic knowledge extracted from Claude Code sessions | 24 | +| [[wiki/concepts/_index\|concepts/]] | Atomic knowledge extracted from Claude Code sessions | 37 | | [[wiki/connections/_index\|connections/]] | Cross-cutting insights linking 2+ concepts | 3 | | [[wiki/qa/_index\|qa/]] | Filed answers to queries (saved with `--file-back`) | 0 | | [[wiki/homelab/_index\|homelab/]] | Self-hosted infra: Proxmox install, IOMMU/PCI passthrough, hypervisor setup, budget builds, Homarr API + Apps + Boards + Certificates + Integrations + Settings + Tasks + AdGuard + Clock + Docker Stats + Docker Integration + Download Client + Firewall + Proxmox Integration + Radarr + Readarr + Sonarr + Bookmarks + Calendar + Icons + App Widget + Weather + GitHub + Nextcloud + qBittorrent + RSS Feed + Speedtest Tracker + System Health Monitoring + System Resources | 35 | @@ -34,6 +34,7 @@ This 3-hop pattern works for hundreds of articles without vector search. | [[wiki/claude-code/_index\|claude-code/]] | Claude Code product docs — install, capabilities, surfaces, MCP, hooks, scheduling, multi-agent, plugins, skills, channels, error recovery | 12 | | [[wiki/reports/_index\|reports/]] | Weekly and monthly knowledge base summaries | 0 | +| [[wiki/infrastructure/_index\|infrastructure/]] | Server inventory: all 10 SSH hosts — optical, optical-dev, baic, librechat, modocmms, box-cli, aimpress, pve | 9 | diff --git a/wiki/concepts/_index.md b/wiki/concepts/_index.md index d7f3bd5..9d8e581 100644 --- a/wiki/concepts/_index.md +++ b/wiki/concepts/_index.md @@ -28,6 +28,21 @@ | [[wiki/concepts/remote-server-dotfiles-bootstrap]] | Full-stack remote dotfiles bootstrap via sshpass r() helper — Fish, Starship, Neovim, CLI tools, fnm, conf.d pattern | daily/2026-04-19.md | 2026-04-19 | | [[wiki/concepts/homarr-proxmox-integration]] | Homarr Proxmox integration cert trust — PVE root CA in trusted-certificates/, Termix SSH REST API, docker-socket-proxy | daily/2026-04-19.md | 2026-04-19 | | [[wiki/concepts/export-endpoint-filter-pattern]] | PDF/CSV exports must receive variant_ids from frontend — frontend-only selection is invisible to backend renderers | daily/2026-04-20.md | 2026-04-20 | +| [[wiki/concepts/ollama-lxc-ram-requirements]] | Ollama in LXC loads Intel oneAPI toolkit (~1.8 GB overhead) even on CPU-only inference — minimum 6 GB RAM for 7B model | daily/2026-04-21.md | 2026-04-21 | +| [[wiki/concepts/beszel-monitoring-deployment]] | Beszel hub + agent deployment across Proxmox host and LXC containers — Ed25519 key auth, port assignment, systemd setup | daily/2026-04-21.md | 2026-04-21 | +| [[wiki/concepts/adguard-dns-rewrites-homelab]] | AdGuard DNS rewrites for *.ai-impress.com wildcard → NPM IP; single entry covers all internal services | daily/2026-04-21.md | 2026-04-21 | +| [[wiki/concepts/prometheus-joules-watts-gotcha]] | rate(joules_total) already returns watts — multiplying by 1000 causes W→kW display bug in Grafana dashboards | daily/2026-04-21.md | 2026-04-21 | +| [[wiki/concepts/old-gpu-sysfs-metrics]] | AMD GCN 1.0 (Oland) and Intel HD 630 don't expose gpu_busy_percent via sysfs — temperatures still available via hwmon | daily/2026-04-21.md | 2026-04-21 | +| [[wiki/concepts/chartjs-time-axis-adapter]] | Chart.js type:time axis requires chartjs-adapter-date-fns registered — without it chart silently fails to render | daily/2026-04-21.md | 2026-04-21 | +| [[wiki/concepts/uptime-kuma-socketio-management]] | Uptime Kuma has no REST API — uses Socket.IO; management done via SQLite directly | daily/2026-04-21.md | 2026-04-21 | +| [[wiki/concepts/proxmox-container-502-misdiagnosis]] | 502 Bad Gateway ≠ dead container — often nginx misconfiguration inside CT; always confirm visually before destructive actions | daily/2026-04-21.md | 2026-04-21 | + +| [[wiki/concepts/git-includeif-per-remote]] | Git includeIf hasconfig:remote.*.url auto-applies correct email per remote — GitHub vs Bitbucket/work without per-repo config | daily/2026-04-23.md | 2026-04-23 | +| [[wiki/concepts/dns-youtube-ad-blocking]] | DNS-based ad blocking cannot block YouTube ads — Google serves ads and content from the same domains; uBlock Origin / ReVanced required | daily/2026-04-23.md | 2026-04-23 | +| [[wiki/concepts/adguard-blocklist-setup]] | AdGuard Home blocklist upgrade: default 1 list → 5 lists (AdAway, HaGeZi Multi Pro++, OISD Big, HaGeZi TIF); ~2.2M domains | daily/2026-04-23.md | 2026-04-23 | + +| [[wiki/concepts/claude-code-schedule-skill-account-type]] | /schedule skill requires claude.ai login account — API-only users need launchd/cron as fallback | daily/2026-04-24.md | 2026-04-24 | +| [[wiki/concepts/memory-compiler-mac-migration]] | Migrating memory-compiler between Mac users — hook PROJECTS_ROOT env var, cc-collector.py hyphen bug, missing reports _index | daily/2026-04-24.md | 2026-04-24 | diff --git a/wiki/concepts/adguard-blocklist-setup.md b/wiki/concepts/adguard-blocklist-setup.md new file mode 100644 index 0000000..5401613 --- /dev/null +++ b/wiki/concepts/adguard-blocklist-setup.md @@ -0,0 +1,109 @@ +--- +title: "AdGuard Home — Blocklist Setup and Optimization" +aliases: [adguard-blocklists, adguard-dns-filter, adguard-hagezi, adguard-oisd, adguard-setup] +tags: [adguard, dns, homelab, ad-blocking, selfhosted, blocklists] +sources: + - "daily/2026-04-23.md" +created: 2026-04-23 +updated: 2026-04-23 +--- + +# AdGuard Home — Blocklist Setup and Optimization + +A fresh AdGuard Home installation ships with a minimal default configuration: one blocklist (AdGuard DNS filter), with AdAway disabled and no custom rules. This covers basic ad blocking but misses a large portion of tracking, malware, and ad domains. Upgrading to 5 curated lists raises coverage from tens of thousands to nearly 2 million domains, significantly improving network-wide blocking for everything DNS-based ad blocking can address. + +## Key Points + +- **Default AdGuard config is intentionally minimal** — 1 list, AdAway disabled; designed to avoid false positives on unknown networks +- **Recommended upgrade: 5 lists total** — AdGuard DNS filter + AdAway + HaGeZi Multi Pro++ + OISD Big + HaGeZi TIF +- **HaGeZi Multi Pro++** (~600k domains) is the best general-purpose upgrade — aggressive but low false-positive rate +- **HaGeZi TIF** (~1M+ domains) targets threat intelligence feeds (malware, phishing, C2) — complements ad blocking with security coverage +- DNS blocklists **cannot block YouTube ads** — see [[wiki/concepts/dns-youtube-ad-blocking]] for why and what works instead + +## Details + +### Recommended Blocklist Stack + +| List | Domains | Focus | Priority | +|------|---------|-------|----------| +| AdGuard DNS filter | ~300k | Ads + tracking (default) | Keep (default) | +| AdAway | ~50k | Mobile ads + analytics | Enable (disabled by default) | +| HaGeZi Multi Pro++ | ~600k | Ads, tracking, cloaking — aggressive | Add | +| OISD Big | ~250k | Broad — ads, privacy, malware | Add | +| HaGeZi TIF | ~1M+ | Threat intelligence: malware, phishing, C2 | Add | + +**Total coverage with all 5:** ~2.2M domains + +### Adding Lists in AdGuard Home UI + +1. Open AdGuard Home → **Filters → DNS blocklists** +2. Click **Add blocklist → Add a custom list** +3. For each list, enter the URL from the table below and click **Save** + +| List | URL | +|------|-----| +| AdAway (enable existing) | Pre-installed — click toggle to enable | +| HaGeZi Multi Pro++ | `https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/multi.txt` | +| OISD Big | `https://big.oisd.nl` | +| HaGeZi TIF | `https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/tif.txt` | + +After adding, AdGuard fetches and compiles all lists. The filter list view shows domain counts per list — confirm all show non-zero counts after a few minutes. + +### HaGeZi Lists Explained + +HaGeZi maintains a tiered family of blocklists with increasing aggressiveness: + +| Tier | Name | Domains | Notes | +|------|------|---------|-------| +| Light | `light.txt` | ~100k | Conservative; low false positives | +| Normal | `normal.txt` | ~300k | General use | +| Pro | `pro.txt` | ~400k | More aggressive | +| **Pro++** | `multi.txt` | ~600k | **Recommended default** | +| Ultimate | `ultimate.txt` | ~700k+ | May break some sites | +| TIF | `tif.txt` | ~1M+ | Threat intelligence only (not general ads) | + +For most home networks, **Multi Pro++** is the right balance. If you see broken sites after adding it, a specific domain can be whitelisted in AdGuard's **Custom filtering rules** without removing the entire list. + +### Whitelisting False Positives + +When a legitimate site is blocked by a blocklist: + +1. AdGuard Home → **Query Log** — find the blocked domain (red entries) +2. Click the entry → **Unblock** — this adds the domain to the whitelist +3. Or manually: **Filters → Custom filtering rules** → add `@@||domain.com^` + +The whitelist takes precedence over all blocklists — a whitelisted domain is never blocked regardless of which list includes it. + +### Monitoring Blocking Effectiveness + +AdGuard Home dashboard shows: +- **DNS Queries:** total queries in time period +- **Blocked:** count and percentage blocked +- **Top blocked domains:** what's being filtered most + +A well-configured home network typically blocks 20–40% of all DNS queries. After upgrading to 5 lists, expect the blocked percentage to increase noticeably (exact amount depends on connected device count and browsing patterns). + +### Checking List Load Status + +After adding lists, verify they loaded correctly: + +``` +AdGuard Home → Filters → DNS blocklists +``` + +Each entry shows: +- Name and URL +- Last update time +- Domain count + +If a list shows 0 domains or "Error fetching", the URL may be stale or the AdGuard instance has no internet access. + +## Related Concepts + +- [[wiki/concepts/dns-youtube-ad-blocking]] — what AdGuard blocklists CAN'T do: block YouTube ads +- [[wiki/concepts/adguard-dns-rewrites-homelab]] — AdGuard's other major use case: split-horizon DNS for internal homelab services +- [[wiki/concepts/tailscale-dns-homelab]] — routing Tailscale clients through AdGuard for network-wide blocking on mobile devices + +## Sources + +- [[daily/2026-04-23.md]] — AdGuard (CT101) had minimal config: 1 list, AdAway disabled; upgraded to 5 lists — enabled AdAway, added HaGeZi Multi Pro++ (~600k), OISD Big (~250k), HaGeZi TIF (~1M+); YouTube ad blocking confirmed impossible via DNS; Piped discussed but not deployed diff --git a/wiki/concepts/adguard-dns-rewrites-homelab.md b/wiki/concepts/adguard-dns-rewrites-homelab.md new file mode 100644 index 0000000..6352925 --- /dev/null +++ b/wiki/concepts/adguard-dns-rewrites-homelab.md @@ -0,0 +1,106 @@ +--- +title: "AdGuard Home — DNS Rewrites for Internal Homelab Services" +aliases: [adguard-dns-rewrites, adguard-internal-dns, adguard-wildcard, homelab-internal-domains] +tags: [adguard, dns, homelab, npm, ssl, wildcard, proxmox, selfhosted] +sources: + - "daily/2026-04-21.md" +created: 2026-04-21 +updated: 2026-04-21 +--- + +# AdGuard Home — DNS Rewrites for Internal Homelab Services + +AdGuard Home's DNS Rewrite feature maps custom domains to internal IP addresses, enabling clean URLs for homelab services (e.g., `grafana.ai-impress.com` → `192.168.1.x`). Combined with a wildcard SSL certificate and an Nginx Proxy Manager (NPM) reverse proxy, this creates a fully functional internal HTTPS stack without exposing services publicly. + +## Key Points + +- **AdGuard DNS Rewrites** are the homelab equivalent of split-horizon DNS — internal domain → internal IP, invisible to public DNS +- Use a **wildcard rewrite** (`*.ai-impress.com → NPM IP`) rather than per-service rewrites — one entry covers all future services automatically +- **NPM is the single ingress point** — AdGuard points all `*.ai-impress.com` to NPM's IP, NPM routes to individual services by hostname +- The wildcard SSL cert (`*.ai-impress.com`) is installed once in NPM and reused for all proxy hosts — no per-service cert management +- **Security model:** internal-only resolution means only devices on LAN or Tailscale VPN can reach these domains; public DNS returns NXDOMAIN for the internal subdomain + +## Details + +### Architecture + +``` +Client (LAN or Tailscale VPN) + │ + ├─ DNS query: grafana.ai-impress.com + │ └─ AdGuard Home: *.ai-impress.com → 192.168.1.225 (NPM) + │ + └─ HTTPS request: grafana.ai-impress.com → 192.168.1.225:443 + └─ NPM: SNI routing → Grafana container (192.168.1.209:3000) + Wildcard cert *.ai-impress.com ✓ +``` + +### Setting Up DNS Rewrites in AdGuard Home + +1. Open AdGuard Home UI → **Filters → DNS Rewrites** +2. Click **Add DNS Rewrite** +3. Domain: `*.ai-impress.com` +4. Answer: `` (e.g., `192.168.1.225`) +5. Save + +This single wildcard entry handles all subdomains. For the bare domain (`ai-impress.com`) pointing to a different IP, add a separate non-wildcard entry. + +### Adding a Proxy Host in NPM + +For each internal service: + +1. NPM UI → **Proxy Hosts → Add Proxy Host** +2. Domain Names: `grafana.ai-impress.com` +3. Forward Hostname/IP: `192.168.1.209` (the container IP) +4. Forward Port: `3000` +5. SSL Certificate: select the existing `*.ai-impress.com` wildcard cert +6. Enable **Force SSL** +7. Save + +The wildcard cert covers all `*.ai-impress.com` without needing individual cert requests. + +### Wildcard SSL Certificate Setup in NPM + +The wildcard cert must be obtained via DNS challenge (not HTTP challenge), since the domain isn't publicly routable. Steps in NPM: + +1. **SSL Certificates → Add SSL Certificate → Let's Encrypt** +2. Domain Names: `*.ai-impress.com` (and optionally `ai-impress.com`) +3. Enable **Use a DNS Challenge** +4. Select DNS provider (Cloudflare, Route53, etc.) and enter API credentials +5. Let's Encrypt issues the wildcard cert via DNS-01 challenge +6. All proxy hosts using this cert auto-renew with NPM's built-in renewal + +### AdGuard Home vs Pi-hole for DNS Rewrites + +| Feature | AdGuard Home | Pi-hole | +|---------|-------------|---------| +| DNS Rewrites UI | Built-in, native | Requires custom HOSTS file or pihole-FTL config | +| Wildcard rewrites | ✓ native | Limited — per-domain only | +| Encrypted DNS (DoH/DoT) | ✓ built-in | Requires unbound sidecar | +| Per-client overrides | ✓ | Limited | +| Service blocking (TikTok, etc.) | ✓ one-click | Manual list | + +AdGuard Home is the correct choice when DNS rewrite management and encrypted DNS are needed alongside ad-blocking. + +### Internal Services with DNS Rewrites (Example Stack) + +| Subdomain | Target | Port | +|-----------|--------|------| +| `homarr.ai-impress.com` | CT107 | 7575 | +| `grafana.ai-impress.com` | CT109 | 3000 | +| `kuma.ai-impress.com` | CT110 | 3001 | +| `vault.ai-impress.com` | CT104 (Vaultwarden) | 80 | +| `proxmox.ai-impress.com` | Proxmox host | 8006 | + +All routed through NPM with the wildcard cert. Only accessible to devices that query AdGuard (LAN or Tailscale with Override local DNS enabled). + +## Related Concepts + +- [[wiki/concepts/tailscale-dns-homelab]] — how Tailscale is configured to use AdGuard as its DNS server, making internal domains accessible on mobile +- [[wiki/concepts/homarr-proxmox-integration]] — Homarr served via this NPM + AdGuard DNS stack +- [[wiki/concepts/nodejs-ssl-system-trust-store]] — SSL trust issues that can arise when services call each other internally +- [[wiki/homelab/_index]] — full homelab infrastructure context + +## Sources + +- [[daily/2026-04-21.md]] — AdGuard DNS rewrites were not configured; setup initiated using wildcard `*.ai-impress.com → NPM IP`; AdGuard Home selected over Pi-hole for built-in DNS rewrite UI, DoH/DoT, and per-client settings diff --git a/wiki/concepts/beszel-monitoring-deployment.md b/wiki/concepts/beszel-monitoring-deployment.md new file mode 100644 index 0000000..06f8ea3 --- /dev/null +++ b/wiki/concepts/beszel-monitoring-deployment.md @@ -0,0 +1,135 @@ +--- +title: "Beszel — Monitoring Hub and Agent Deployment Across Proxmox + LXC" +aliases: [beszel, beszel-agent, beszel-deployment, beszel-proxmox] +tags: [beszel, monitoring, proxmox, lxc, homelab, systemd, metrics] +sources: + - "daily/2026-04-21.md" +created: 2026-04-21 +updated: 2026-04-21 +--- + +# Beszel — Monitoring Hub and Agent Deployment Across Proxmox + LXC + +Beszel is a lightweight server monitoring tool with a hub-and-agent architecture. The hub runs as a single container and collects metrics from `beszel-agent` binaries deployed on each host. Agents authenticate using the hub's Ed25519 public key — not tokens or passwords. Deploying agents across a Proxmox host and all LXC containers requires installing the binary and creating a systemd service on each. + +## Key Points + +- **Beszel shows no systems until `beszel-agent` is installed** — the hub UI is empty by default; agents must be deployed separately on each target host +- **Authentication uses Ed25519 public key** from the hub — obtain from Beszel UI when adding a new system; not a JWT or API token +- **Each agent gets its own port** — Proxmox host typically on 45876; LXC containers on 45877–45882 (sequential); must be unique per host +- **"System shows red" usually means wrong IP/port in the hub UI**, not a broken agent — verify the agent is listening before debugging the agent itself +- **CT102 (Docker host) exposes inner container metrics** to Beszel via socket access, not separate per-container agents + +## Details + +### Architecture + +``` +Beszel Hub (PocketBase-based) + ├── port 8090 (web UI + API) + └── Agents push metrics via Ed25519-secured connection + ├── Proxmox host → beszel-agent :45876 + ├── CT100 (Ollama) → beszel-agent :45877 + ├── CT102 (Docker) → beszel-agent :45878 (+ Docker socket for container metrics) + ├── CT105 (NPM) → beszel-agent :45879 + ├── CT107 (Homarr) → beszel-agent :45880 + ├── CT109 (Grafana) → beszel-agent :45881 + └── CT110 (Uptime Kuma) → beszel-agent :45882 +``` + +### Agent Installation (Each Host) + +```bash +# Download and install beszel-agent binary +curl -sL https://raw.githubusercontent.com/henrygd/beszel/main/supplemental/scripts/install-agent.sh | bash -s -- \ + -p 45876 \ + -k "ed25519 public key from hub" + +# The install script creates a systemd service automatically +systemctl status beszel-agent +``` + +Replace `45876` with the appropriate port for each container. The Ed25519 public key is shown in the Beszel hub UI when you click "Add System". + +### Manual Systemd Service (Alternative) + +If the install script is unavailable or fails: + +```ini +# /etc/systemd/system/beszel-agent.service +[Unit] +Description=Beszel Agent +After=network-online.target + +[Service] +ExecStart=/usr/local/bin/beszel-agent +Environment=PORT=45876 +Environment=KEY="ed25519 AAAA..." +Restart=always +RestartSec=5 + +[Install] +WantedBy=multi-user.target +``` + +```bash +systemctl daemon-reload +systemctl enable --now beszel-agent +``` + +### CT102 — Docker Socket Integration + +For the container that runs Docker Compose services, mount the Docker socket into the beszel-agent to expose inner container metrics (CPU, memory per container): + +```bash +# Add to beszel-agent environment +Environment=DOCKER_HOST=unix:///var/run/docker.sock +``` + +Or use docker-socket-proxy if exposing the raw socket is a concern. + +### Adding Systems in the Hub UI + +After deploying agents, add each system in `http://:8090/`: + +1. Click **Add System** +2. Enter Name, Host (IP of the container/host), Port (the agent port) +3. Copy the **Ed25519 public key** shown in the dialog — paste this into the agent's `KEY` environment variable +4. Save + +If a system shows **red** after adding: +- First check the IP and port in the UI — this is the most common cause +- `curl http://:/healthz` from another container to confirm connectivity +- Only then check the agent's systemd logs + +### Diagnosing "System Shows Red" + +```bash +# On the hub or another container — check agent is reachable +curl http://192.168.1.127:45877/ + +# On the agent container — check it's listening +ss -tlnp | grep 45877 + +# Check agent logs +journalctl -u beszel-agent -n 20 + +# If agent is healthy but hub shows red → fix IP/port in hub UI +``` + +The 2026-04-21 incident: Ollama (CT100) showed red in Beszel despite a healthy agent at `192.168.1.127:45877`. Root cause was the hub UI entry had the wrong IP — corrected to `.127` from `.xxx`, and Ollama turned green immediately. + +### Temperature Variance Across Containers + +All LXC containers on the same Proxmox host read the **same physical temperature sensors** (CPU cores, NVMe, board). Temperature readings may differ by ±5–10°C across containers due to different polling timestamps — this is normal polling jitter, not a hardware issue. + +## Related Concepts + +- [[wiki/concepts/ollama-lxc-ram-requirements]] — Ollama LXC container context where Beszel was used to monitor RAM usage +- [[wiki/concepts/homarr-proxmox-integration]] — Homarr is another monitoring/dashboard layer alongside Beszel +- [[wiki/concepts/proxmox-mcp-server]] — Proxmox MCP for managing the underlying infrastructure +- [[wiki/homelab/_index]] — full homelab infrastructure context + +## Sources + +- [[daily/2026-04-21.md]] — Beszel showed empty UI because no agents were installed; deployed beszel-agent on Proxmox host (45876) and 6 LXC containers (45877–45882); CT102 Docker socket integration; Ollama showed red due to wrong IP in hub UI (`.xxx` vs `.127`); temperature variance explained diff --git a/wiki/concepts/chartjs-time-axis-adapter.md b/wiki/concepts/chartjs-time-axis-adapter.md new file mode 100644 index 0000000..1525502 --- /dev/null +++ b/wiki/concepts/chartjs-time-axis-adapter.md @@ -0,0 +1,104 @@ +--- +title: "Chart.js — type:time Axis Requires a Date Adapter" +aliases: [chartjs-time-axis, chartjs-date-adapter, chartjs-adapter-date-fns, chartjs-silent-failure] +tags: [chartjs, javascript, frontend, dashboard, time-series, debugging] +sources: + - "daily/2026-04-21.md" +created: 2026-04-21 +updated: 2026-04-21 +--- + +# Chart.js — type:time Axis Requires a Date Adapter + +Chart.js does not bundle a date/time parsing library. When a chart uses `type: 'time'` on an axis (common for time-series dashboards), Chart.js requires an external date adapter to be registered. Without it, the chart initialization crashes silently in JavaScript before any data is rendered — the canvas element renders blank with no visible error in the chart itself. + +## Key Points + +- **`type: 'time'` axis fails silently without a registered date adapter** — no visible error in the chart; JavaScript throws internally but the chart just renders blank +- **`chartjs-adapter-date-fns`** is the most common adapter — requires importing both `chart.js` and `chartjs-adapter-date-fns`; the adapter registers itself on import +- The blank canvas symptom is identical to "no data" — always check for a missing adapter before debugging data fetching or chart config +- **`chart.js/auto` import is NOT sufficient** — the auto bundle includes Chart.js with all chart types but still excludes the date adapter +- The adapter must be imported before `new Chart(...)` is called; it registers itself globally via side effect + +## Details + +### The Failure Mode + +```js +// Chart config with time axis +const config = { + type: 'line', + data: { datasets: [/* ... */] }, + options: { + scales: { + x: { + type: 'time', // ← requires date adapter + time: { unit: 'hour' } + } + } + } +}; + +new Chart(ctx, config); +// If adapter is missing: Chart.js throws internally +// "Error: This method is not implemented: Check that a complete date adapter is provided" +// The canvas stays blank — no user-visible error if exception is caught silently +``` + +### The Fix + +```js +// 1. Install the adapter +// npm install chartjs-adapter-date-fns date-fns +// or +// + +// 2. Import it before using Chart.js — it self-registers +import 'chartjs-adapter-date-fns'; +import { Chart } from 'chart.js'; + +// Now type: 'time' works correctly +new Chart(ctx, config); +``` + +For CDN / script tag usage (no bundler): +```html + + + + +``` + +### Why the Failure Is Silent + +Chart.js catches the adapter error internally and attempts to degrade gracefully. The error message `"This method is not implemented"` may appear in the browser console (F12 → Console), but: +1. Dashboard apps often have global error handlers that suppress console output +2. The chart canvas is present and sized correctly — it just has no content +3. It looks identical to "data is empty" or "data fetch failed" + +Always open browser DevTools → Console before assuming the data pipeline is broken. + +### Alternative Adapters + +| Adapter | Library | Bundle Size | +|---------|---------|-------------| +| `chartjs-adapter-date-fns` | date-fns | ~8 KB (bundled) | +| `chartjs-adapter-luxon` | Luxon | ~14 KB | +| `chartjs-adapter-moment` | Moment.js | ~30 KB+ | +| `chartjs-adapter-dayjs` | Day.js | ~4 KB | + +`chartjs-adapter-date-fns` is the standard recommendation for new projects — date-fns is tree-shakable and the combined bundle is small. + +### Context: Power & Cost Dashboard (2026-04-21) + +A homelab monitoring dashboard used Chart.js with `type: 'time'` on the X axis to display power consumption over time. The `refresh()` function that fetched data from Prometheus never ran because the chart initialization threw before reaching it. The symptom was a completely blank dashboard — no charts, no errors displayed. Adding `chartjs-adapter-date-fns` resolved the initialization crash and all charts appeared. + +## Related Concepts + +- [[wiki/concepts/prometheus-joules-watts-gotcha]] — the other Chart.js dashboard bug found in the same session (wrong units) +- [[wiki/concepts/beszel-monitoring-deployment]] — Beszel as a monitoring alternative that doesn't require Chart.js setup +- [[wiki/homelab/_index]] — homelab monitoring stack context + +## Sources + +- [[daily/2026-04-21.md]] — Power & Cost dashboard blank due to Chart.js `type: 'time'` crash; missing `chartjs-adapter-date-fns`; adding the adapter resolved the initialization crash and data loaded correctly diff --git a/wiki/concepts/claude-code-schedule-skill-account-type.md b/wiki/concepts/claude-code-schedule-skill-account-type.md new file mode 100644 index 0000000..c7c3b61 --- /dev/null +++ b/wiki/concepts/claude-code-schedule-skill-account-type.md @@ -0,0 +1,111 @@ +--- +title: "Claude Code — /schedule Skill Requires claude.ai Login Account" +aliases: [claude-code-schedule, schedule-skill-auth, launchd-claude-code, cron-claude-code] +tags: [claude-code, scheduling, launchd, macos, automation, knowledge-base] +sources: + - "daily/2026-04-24.md" +created: 2026-04-24 +updated: 2026-04-24 +--- + +# Claude Code — /schedule Skill Requires claude.ai Login Account + +The `/schedule` skill in Claude Code creates recurring remote agent triggers that run on a cron schedule. It requires a **claude.ai login account** — it does not work with API-only accounts (accounts authenticated purely via `ANTHROPIC_API_KEY` without a claude.ai web account). On macOS, the fallback for periodic automation is native `launchd` Launch Agents, which run shell commands on a schedule without requiring a Claude login. + +## Key Points + +- **`/schedule` requires a claude.ai account** (browser login), not an API-only account — attempting it with API-only auth results in a failure or unsupported operation +- **macOS launchd** is the correct fallback: Launch Agent plist files in `~/Library/LaunchAgents/` with `StartCalendarInterval` for cron-like scheduling +- launchd persists across reboots, respects user session, and can run any shell command — including `uv run python scripts/compile.py` for knowledge base automation +- A **weekly** launchd job (e.g., Mondays 09:00) and a **monthly** job (1st of month, 09:00) are the two schedules needed for knowledge base report generation +- The launchd approach requires no Claude authentication — it runs independently as a macOS system service + +## Details + +### Why /schedule Doesn't Work for API Accounts + +The `/schedule` skill stores trigger configurations against the authenticated user's claude.ai session. The underlying API for managing scheduled remote agents is tied to the web account identity, not the API key. Developers who use Claude Code primarily via the terminal with an `ANTHROPIC_API_KEY` (without having set up a claude.ai web session) cannot create or list scheduled triggers. + +This limitation is documented in the skill's description: "Create, update, list, or run scheduled remote agents (triggers) that execute on a cron schedule." The agent execution model requires the claude.ai account to host the trigger state. + +### macOS launchd as Fallback + +macOS Launch Agents (`~/Library/LaunchAgents/`) are the native equivalent of user-level cron jobs. They are more robust than cron: they survive reboots, support calendar-based scheduling, and log to the system log. + +**Weekly report plist** (`~/Library/LaunchAgents/com.memory-compiler.weekly-report.plist`): + +```xml + + + + + Label + com.memory-compiler.weekly-report + ProgramArguments + + /bin/bash + -c + cd ~/.claude/memory-compiler && uv run python scripts/report-generator.py --weekly + + StartCalendarInterval + + Weekday + 1 + Hour + 9 + Minute + 0 + + StandardOutPath + /tmp/memory-compiler-weekly.log + StandardErrorPath + /tmp/memory-compiler-weekly.err + + +``` + +**Monthly report plist** — same structure with `StartCalendarInterval` using `Day: 1` instead of `Weekday: 1`. + +### Loading / Managing launchd Jobs + +```bash +# Load (activate) a job +launchctl load ~/Library/LaunchAgents/com.memory-compiler.weekly-report.plist + +# Unload (deactivate) a job +launchctl unload ~/Library/LaunchAgents/com.memory-compiler.weekly-report.plist + +# Check if loaded +launchctl list | grep memory-compiler + +# Force-run immediately (for testing) +launchctl start com.memory-compiler.weekly-report + +# View logs +tail -f /tmp/memory-compiler-weekly.log +``` + +The job must be loaded after creation and after each reboot (or set `RunAtLoad: true` if you want it to also run on load). + +### When to Use /schedule vs launchd + +| Scenario | Use | Why | +|----------|-----|-----| +| claude.ai login account available | `/schedule` skill | Managed by Claude, visible in UI, no local config | +| API-only account | launchd (macOS) / cron (Linux) | No claude.ai auth required; runs locally | +| Tasks that need Claude Code itself | `/schedule` | Triggers a remote Claude agent | +| Tasks running a local script | launchd | No need for Claude; just shell commands | +| Cross-machine scheduling | `/schedule` | Stored in cloud, fires from anywhere | + +For knowledge base compilation and report generation (local Python scripts), launchd is the correct tool regardless of account type — the scripts don't need Claude Code to run them, they use the Claude API directly. + +## Related Concepts + +- [[wiki/claude-code/scheduled-tasks]] — `/loop`, `CronCreate/List/Delete` tools, and the `/schedule` skill documentation +- [[wiki/concepts/memory-compiler-mac-migration]] — the migration session where the `/schedule` limitation was discovered +- [[wiki/claude-code/headless-cli]] — running Claude Code non-interactively, which also involves auth considerations + +## Sources + +- [[daily/2026-04-24.md]] — Mac migration session: `/schedule` skill invoked but failed because account is API-only; launchd selected as alternative; weekly (Monday 09:00) and monthly (1st of month) launchd plists created for `report-generator.py` diff --git a/wiki/concepts/dns-youtube-ad-blocking.md b/wiki/concepts/dns-youtube-ad-blocking.md new file mode 100644 index 0000000..117c700 --- /dev/null +++ b/wiki/concepts/dns-youtube-ad-blocking.md @@ -0,0 +1,91 @@ +--- +title: "DNS Ad Blocking — YouTube Ads Cannot Be Blocked via DNS" +aliases: [youtube-dns-blocking, adguard-youtube, pihole-youtube, dns-youtube-limits] +tags: [adguard, pihole, dns, youtube, ad-blocking, homelab, android] +sources: + - "daily/2026-04-23.md" +created: 2026-04-23 +updated: 2026-04-23 +--- + +# DNS Ad Blocking — YouTube Ads Cannot Be Blocked via DNS + +DNS-based ad blocking (AdGuard Home, Pi-hole, NextDNS) works by refusing to resolve domains known to serve ads. YouTube ads are served from the same domains as regular YouTube content — `youtube.com`, `googlevideo.com`, `yt3.ggpht.com` — making it impossible to block ads without blocking all of YouTube. This is intentional on Google's part and has been true since approximately 2023. The only effective solutions require client-side interception. + +## Key Points + +- **YouTube ad domains = YouTube content domains** — blocking `googlevideo.com` prevents video playback entirely, not just ads +- DNS ad blocking **cannot distinguish** between an ad video stream and a regular video stream at the DNS resolution level +- This is Google's deliberate countermeasure against DNS-based blocking — they co-locate ad serving infrastructure with content CDN +- **Working solutions require client-side interception:** uBlock Origin in Firefox, YouTube ReVanced (Android), SmartTubeNext (Android TV) +- **Piped is more viable than Invidious** for self-hosted YouTube frontends — Google actively blocks IP ranges used by Invidious instances + +## Details + +### Why DNS Blocking Fails for YouTube + +DNS ad blocking works well for domains that exist solely to serve ads (e.g., `ad.doubleclick.net`, `track.googleadservices.com`). When your browser requests `ad.doubleclick.net`, AdGuard returns NXDOMAIN — the request never goes out, the ad never loads. + +YouTube ads are different: they're served as DASH/HLS video segments from the same `googlevideo.com` CDN that delivers the actual video content. The ad segment URL and the content segment URL are structurally identical: + +``` +# Ad segment (same domain, same path structure as real content) +https://rr1---sn-xxx.googlevideo.com/videoplayback?id=xyz&...&oad=1 + +# Video content segment +https://rr1---sn-xxx.googlevideo.com/videoplayback?id=xyz&... +``` + +DNS sees only `googlevideo.com` — blocking it stops all YouTube video playback. Not blocking it allows ads to load. There is no middle ground at the DNS layer. + +### Community Consensus (2025–2026) + +This is not a gap in blocklist coverage or a matter of finding the right filters. It is an architectural limitation. AdGuard, Pi-hole, and NextDNS all acknowledge they cannot block YouTube ads. Adding more blocklists does not help — the issue is not missing filter rules but that the correct filter rule (`block googlevideo.com`) would be catastrophic. + +### Effective Solutions by Platform + +| Platform | Solution | Notes | +|----------|---------|-------| +| Desktop (Firefox) | uBlock Origin | Most effective; blocks via content script injection | +| Desktop (Chrome/Chromium) | uBlock Origin | Works; Manifest V3 transition ongoing | +| Android phone | YouTube ReVanced | Patched YouTube APK; requires manual install | +| Android TV | SmartTubeNext | Open-source, ad-free, supports 4K | +| iPhone/iPad | Limited — no sideloading | Brave browser has partial blocking | +| Smart TV (native app) | None | No extension/sideloading support | + +### Self-Hosted YouTube Frontends: Piped vs Invidious + +For users who want a completely ad-free YouTube experience with no client-side app: + +**Invidious** — self-hosted YouTube frontend that proxies YouTube content +- Google has been aggressively blocking IP ranges used by Invidious instances since 2023 +- Public Invidious instances frequently break (videos fail to load) +- Self-hosted instances on residential/VPS IPs are less targeted but not immune + +**Piped** — alternative self-hosted YouTube frontend +- Better maintained as of 2026; more resilient to Google's blocking +- Supports RSS feeds, playlists, SponsorBlock integration +- Still subject to YouTube breaking changes, but Google has been less aggressive about blocking Piped IPs + +Neither solution is a reliable long-term substitute for a YouTube subscription or a client-side blocker. Self-hosted frontends are technically complex to maintain. + +### What DNS Blocking CAN Reduce + +While DNS blocking can't stop YouTube ads, it does meaningfully reduce: +- Google Analytics tracking across all sites +- Facebook pixel tracking +- Most banner and interstitial ads on non-Google sites +- Telemetry pings from apps +- Crypto mining scripts + +A well-configured AdGuard (5+ blocklists) blocks 30–50% of all DNS queries on a typical home network — just not YouTube ads specifically. + +## Related Concepts + +- [[wiki/concepts/adguard-blocklist-setup]] — recommended blocklists for maximizing AdGuard effectiveness on everything it CAN block +- [[wiki/concepts/adguard-dns-rewrites-homelab]] — AdGuard's other main use case: internal DNS rewrites for homelab services +- [[wiki/concepts/tailscale-dns-homelab]] — Tailscale DNS setup to route mobile DNS through AdGuard for network-wide blocking on mobile + +## Sources + +- [[daily/2026-04-23.md]] — AdGuard Home session: YouTube ads not blocked despite AdGuard being active; diagnosis confirmed DNS-level blocking is architecturally impossible for YouTube; solutions identified: uBlock Origin (Firefox), YouTube ReVanced (Android), SmartTubeNext (Android TV); Piped preferred over Invidious for self-hosted frontend; decision to not set up Piped/Invidious diff --git a/wiki/concepts/git-includeif-per-remote.md b/wiki/concepts/git-includeif-per-remote.md new file mode 100644 index 0000000..f139c0d --- /dev/null +++ b/wiki/concepts/git-includeif-per-remote.md @@ -0,0 +1,111 @@ +--- +title: "Git — Per-Remote Identity with includeIf" +aliases: [git-includeif, git-multiple-identities, git-work-personal-email, git-conditional-config] +tags: [git, dotfiles, identity, ssh, bitbucket, github] +sources: + - "daily/2026-04-23.md" +created: 2026-04-23 +updated: 2026-04-23 +--- + +# Git — Per-Remote Identity with includeIf + +Git's `includeIf` directive in `.gitconfig` applies a supplemental config file only when a condition matches. The `hasconfig:remote.*.url` variant matches on the remote URL of the current repository, making it possible to automatically switch git identities (email, signing key, etc.) based on whether a repo is hosted on GitHub vs Bitbucket or any other remote — without any per-repo manual config. + +## Key Points + +- **`includeIf "hasconfig:remote.*.url:*github.com*"`** matches any repo with a GitHub remote — applies a separate `.gitconfig-github` file automatically +- Eliminates the need to run `git config user.email` in every new personal repo — the correct identity is applied by remote URL +- The condition uses glob-style wildcards: `*github.com*` matches both `https://github.com/...` and `git@github.com:...` +- More robust than `includeIf "gitdir:~/projects/personal/"` (directory-based) — works regardless of where the repo is cloned +- Order matters: the included file is applied after the main config, so it overrides values in `.gitconfig` + +## Details + +### The Problem This Solves + +Working with multiple git identities (work email for Bitbucket/Oliver repos, personal email for GitHub) previously required remembering to run `git config user.email` in each new personal repo. Forgetting resulted in commits to GitHub attributed to the work email. Remote-based `includeIf` makes this automatic and invisible. + +### `.gitconfig` Setup + +```ini +# ~/.gitconfig — main config (default = work identity) +[user] + name = Vadym Samoilenko + email = vadymsamoilenko@oliver.agency + +# Override with personal identity for GitHub repos +[includeIf "hasconfig:remote.*.url:*github.com*"] + path = ~/.gitconfig-github +``` + +```ini +# ~/.gitconfig-github — personal identity +[user] + email = samoylenko.vadym@gmail.com +``` + +After this setup, any repo with a `github.com` remote automatically uses the personal email. Bitbucket, internal GitLab, and other remotes use the default work email. + +### Verifying the Correct Identity + +```bash +# In a GitHub repo +git config user.email +# → samoylenko.vadym@gmail.com + +# In a Bitbucket/work repo +git config user.email +# → vadymsamoilenko@oliver.agency +``` + +### SSH Key Separation (Complementary) + +`includeIf` handles commit identity (who authored the commit). SSH key selection (which credential is used to authenticate the push) is handled separately in `~/.ssh/config`: + +```ssh +# ~/.ssh/config +Host github.com + HostName github.com + User git + IdentityFile ~/.ssh/id_ed25519_github + +Host bitbucket.org + HostName bitbucket.org + User git + IdentityFile ~/.ssh/id_ed25519_bitbucket +``` + +Both pieces are needed: `includeIf` for commit metadata, `~/.ssh/config` for authentication. + +### Alternative: Directory-Based includeIf + +If the remote URL approach is too broad, a directory-based condition is more explicit: + +```ini +[includeIf "gitdir:~/personal/"] + path = ~/.gitconfig-github +``` + +This applies the override only to repos inside `~/personal/`. The downside: repos cloned anywhere else won't get the override automatically. + +### Migration: Adding includeIf to an Existing Config + +When migrating from a single identity to multiple identities: + +1. Set the work email as the default in `~/.gitconfig` +2. Create `~/.gitconfig-github` with the personal email +3. Add the `includeIf` block to `~/.gitconfig` +4. Verify in an existing GitHub repo: `git config user.email` + +Past commits already made with the wrong email are not affected — `includeIf` only applies to future commits. + +## Related Concepts + +- [[wiki/concepts/fish-shell-path-config]] — SSH aliases as Fish functions complement this SSH config approach +- [[wiki/concepts/remote-server-dotfiles-bootstrap]] — dotfiles migration context where this pattern was configured +- [[wiki/concepts/bitbucket-mcp-atlassian]] — Bitbucket identity context (one of the two remotes managed by this config) + +## Sources + +- [[daily/2026-04-23.md]] — New workstation SSH/git migration session: git identity split required between `vadymsamoilenko@oliver.agency` (Bitbucket/work) and `samoylenko.vadym@gmail.com` (GitHub); `includeIf "hasconfig:remote.*.url:*github.com*"` selected as the mechanism diff --git a/wiki/concepts/memory-compiler-mac-migration.md b/wiki/concepts/memory-compiler-mac-migration.md new file mode 100644 index 0000000..631f4c0 --- /dev/null +++ b/wiki/concepts/memory-compiler-mac-migration.md @@ -0,0 +1,146 @@ +--- +title: "Memory Compiler — Migrating Between Mac Users" +aliases: [memory-compiler-migration, cc-memory-migration, claude-code-path-migration] +tags: [memory-compiler, claude-code, hooks, macos, migration, knowledge-base] +sources: + - "daily/2026-04-24.md" +created: 2026-04-24 +updated: 2026-04-24 +--- + +# Memory Compiler — Migrating Between Mac Users + +Migrating the Claude Code memory-compiler system from one Mac user account to another (or between machines with different usernames) requires several manual fixes that are not obvious from the README. The hooks, `cc-collector.py`, and the wiki structure all have hardcoded or derived path assumptions that break on a new user path. Missing any of these produces silent failures — sessions appear to flush correctly but no data reaches the knowledge base. + +## Key Points + +- **All 3 hooks** (SessionStart, PreCompact, SessionEnd) must have `PROJECTS_ROOT` explicitly set as an environment variable — if missing, the obsidian session-start script silently produces empty context +- **`cc-collector.py` has a folder name bug**: `_root_prefix()` derives the project identifier using underscores, but the session storage folder name uses hyphens (e.g., `ai-leed` not `ai_leed`) — the fix is to ensure the prefix conversion maps `_` → `-` +- **`wiki/reports/_index.md`** must exist for the reports topic to appear in the master index — it is not auto-created and must be created manually when setting up a fresh wiki +- **`projects-overview` article count** must be updated in the master index after migration (was 36 on old Mac; 37 on new Mac after project was added) +- The old Mac's files are accessible via SSH if needed — no need to manually recreate plugin caches or config files + +## Details + +### Hook Environment Variables (PROJECTS_ROOT) + +The obsidian hooks (`obsidian-session-start.py`, `obsidian-pre-compact.py`, `obsidian-stop.py`) read a `PROJECTS_ROOT` environment variable to find project directories. On the old Mac this was set in the hook command line; when migrating, it must be re-added to the global `~/.claude/settings.json` for each hook. + +The format in `~/.claude/settings.json`: + +```json +{ + "hooks": { + "SessionStart": [ + { + "hooks": [{ + "type": "command", + "command": "PROJECTS_ROOT=/Users/ai_leed/Projects python3 ~/.claude/obsidian-session-start.py ...", + "statusMessage": "Loading Obsidian context..." + }] + } + ] + } +} +``` + +If `PROJECTS_ROOT` is missing, the hook runs but finds no projects — `additionalContext` is empty. The session proceeds without any Obsidian note context injected, and no error appears. This is the silent failure mode. + +### cc-collector.py Folder Name Bug + +The `cc-collector.py` script (part of the Claude Code dashboard collector) uses `_root_prefix()` to derive a project identifier from the project directory path. The function was converting the macOS username portion of the path to an identifier using underscores: `/Users/ai_leed/...` → `ai_leed_...`. + +However, Claude Code stores session data in folders named with hyphens: `ai-leed` not `ai_leed`. The mismatch meant `cc-collector.py` was looking for session files in a folder that didn't exist. + +**Fix applied to `_root_prefix()`:** +```python +def _root_prefix(path: str) -> str: + # Convert path separators to hyphens, not underscores + # Folder names use hyphens: /Users/ai_leed -> ai-leed (not ai_leed) + parts = path.strip("/").split("/") + return "-".join(parts).replace("_", "-") +``` + +The general pattern: whenever deriving a folder name from a user path, use hyphens throughout. + +### Creating wiki/reports/_index.md + +The reports section of the master index (`| [[wiki/reports/_index\|reports/]] | ... | 0 |`) requires the file to exist. A fresh wiki setup does not create it automatically. + +Minimal file to create at `wiki/reports/_index.md`: + +```markdown +# Reports — Topic Index + +> Weekly and monthly knowledge base summaries generated by report-generator.py. +> Compiled automatically via launchd on Mondays (weekly) and the 1st of each month (monthly). + +| Report | Period | Generated | +|--------|--------|-----------| + + +``` + +Without this file, links in the master index are broken wikilinks that Obsidian highlights in red. + +### Migration Checklist + +Complete checklist when migrating the memory-compiler to a new Mac user: + +``` +[ ] 1. Clone/copy memory-compiler repo to ~/.claude/memory-compiler +[ ] 2. Run: cd ~/.claude/memory-compiler && uv sync +[ ] 3. Update all hook commands in ~/.claude/settings.json with new username paths +[ ] 4. Add PROJECTS_ROOT=... env var to each obsidian hook command +[ ] 5. Verify cc-collector.py _root_prefix() uses hyphens not underscores +[ ] 6. Create wiki/reports/_index.md if missing +[ ] 7. Update projects-overview article count in _master-index.md if it changed +[ ] 8. Run a manual compile to verify pipeline: uv run python scripts/compile.py --dry-run +[ ] 9. Set up launchd jobs for weekly/monthly reports (if /schedule skill unavailable) +[ ] 10. Verify SessionStart hook fires: open a new Claude Code session, check additionalContext +``` + +### Identifying Old Mac Files via SSH + +If config files, plugin caches, or other assets exist on the old Mac under `/Users/aimpress/` and are needed, they can be fetched via SSH: + +```bash +# Fetch a specific file +scp aimpress@:/Users/aimpress/.claude/settings.json /tmp/old-settings.json + +# Or rsync a directory +rsync -avz aimpress@:/Users/aimpress/.claude/plugins/ ~/.claude/plugins/ +``` + +There is no need to recreate plugin caches manually — they rebuild on first use. + +### Verifying the Pipeline After Migration + +After completing the migration checklist, verify each layer: + +```bash +# 1. Test hooks fire (open Claude Code, look for "Loading knowledge context..." status) +# 2. Test flush works +cd ~/.claude/memory-compiler +echo "test session" > /tmp/test-flush.md +uv run python scripts/flush.py /tmp/test-flush.md + +# 3. Test compile works +uv run python scripts/compile.py --dry-run + +# 4. Check logs +tail -20 scripts/flush.log +tail -20 scripts/compile.log +``` + +A successful flush appends a `FLUSH_OK` line or bullet points to `daily/YYYY-MM-DD.md`. A failed flush appends `FLUSH_ERROR: Exception: Command failed with exit code 1`. + +## Related Concepts + +- [[wiki/concepts/claude-code-schedule-skill-account-type]] — the `/schedule` skill limitation discovered during this migration session; launchd used as fallback +- [[wiki/claude-code/overview]] — Claude Code hook system overview including SessionStart, PreCompact, SessionEnd +- [[wiki/obsidian-rag/_index]] — the Karpathy LLM wiki method that the memory-compiler implements + +## Sources + +- [[daily/2026-04-24.md]] — Mac migration session (/Users/aimpress/ → /Users/ai_leed/): 3 hooks missing PROJECTS_ROOT env var (all fixed); cc-collector.py _root_prefix() underscore→hyphen bug found and fixed; wiki/reports/_index.md created; projects-overview count updated 36→37; 6 daily logs (Apr 19–24) compiled manually after migration diff --git a/wiki/concepts/old-gpu-sysfs-metrics.md b/wiki/concepts/old-gpu-sysfs-metrics.md new file mode 100644 index 0000000..a0beb7c --- /dev/null +++ b/wiki/concepts/old-gpu-sysfs-metrics.md @@ -0,0 +1,116 @@ +--- +title: "Old GPU Sysfs Metrics — AMD GCN 1.0 and Intel iGPU Limitations" +aliases: [gpu-sysfs-metrics, amd-oland-metrics, gcn1-metrics, intel-igpu-metrics, gpu-busy-percent] +tags: [gpu, amd, intel, sysfs, prometheus, grafana, homelab, metrics, monitoring] +sources: + - "daily/2026-04-21.md" +created: 2026-04-21 +updated: 2026-04-21 +--- + +# Old GPU Sysfs Metrics — AMD GCN 1.0 and Intel iGPU Limitations + +AMD GCN 1.0 generation GPUs (codenamed Oland, Cape Verde, Pitcairn, etc., circa 2012–2013) and Intel HD 600-series integrated GPUs do not expose `gpu_busy_percent` through the Linux sysfs interface. These chips predate the kernel driver support for hardware utilization counters. Temperature and fan metrics are still available via hwmon, but GPU utilization percentage cannot be collected without specialized tools. + +## Key Points + +- **AMD Oland (GCN 1.0) does not expose `gpu_busy_percent`** via `/sys/class/drm/card*/device/gpu_busy_percent` — the sysfs file does not exist +- **Intel HD 630 and similar iGPUs** also lack sysfs utilization exposure; Intel's metrics are only accessible via `intel_gpu_top` (requires root) or vendor-specific interfaces +- **Temperatures are still available** via hwmon: CPU cores, NVMe, board temperature are exposed regardless of GPU generation +- **Textfile collector** is the correct long-term approach — write a shell script to collect GPU metrics via available tools (e.g., `amdgpu_top`, `radeontop`) and expose them as Prometheus metrics via `node_exporter`'s `--collector.textfile` flag +- At the time of the 2026-04-21 session, the textfile collector infrastructure was set up but the cron job was not activated (pending confirmation) + +## Details + +### Why gpu_busy_percent Is Missing on GCN 1.0 + +The `gpu_busy_percent` sysfs attribute is provided by the `amdgpu` kernel driver starting with GCN 2.0 hardware (Bonaire/Hawaii, 2013+). GCN 1.0 cards (Oland, Pitcairn, Tahiti) use the same driver but the hardware performance counter interface is not implemented for that silicon generation. The file simply doesn't exist in sysfs. + +Checking: +```bash +# This path does not exist on GCN 1.0 +cat /sys/class/drm/card0/device/gpu_busy_percent +# cat: /sys/class/drm/card0/device/gpu_busy_percent: No such file or directory + +# This DOES work — temperature via hwmon +cat /sys/class/hwmon/hwmon*/temp*_input +``` + +### Intel iGPU (HD 600 Series) + +Intel HD 630 (Kaby Lake) and similar integrated GPUs expose minimal sysfs data. Intel GPU utilization requires either: +- `intel_gpu_top` — requires root, outputs to terminal, not easily scriptable for Prometheus +- `/sys/class/drm/card*/gt/gt0/rc6_residency_ms` — residency counter, not utilization percentage +- Intel GVT-g GPU virtualization layer — complex setup, not appropriate for simple monitoring + +For a homelab, accepting that Intel iGPU utilization won't be in dashboards is the pragmatic choice. + +### What IS Available (All GPU Generations) + +Even on old hardware, these metrics are typically available: + +```bash +# GPU temperature (amdgpu hwmon) +/sys/class/hwmon/hwmon*/temp1_input # GPU die temp (millidegrees C) + +# Fan speed (if applicable) +/sys/class/hwmon/hwmon*/fan1_input # RPM + +# Clock frequencies (may be available) +/sys/class/hwmon/hwmon*/freq1_input # GPU clock Hz +``` + +In Grafana: CPU core temps, NVMe temp, board temp, and fan RPM can all be scraped via `node_exporter` and displayed in a System Overview dashboard even without GPU utilization %. + +### Textfile Collector Pattern for Future GPUs + +When the homelab gets a newer GPU (GCN 2.0+ or NVIDIA), use the textfile collector to expose metrics: + +```bash +# /usr/local/bin/gpu-metrics.sh +#!/bin/bash +# Scrapes AMD GPU utilization and writes Prometheus-format metrics + +GPU_BUSY=$(cat /sys/class/drm/card0/device/gpu_busy_percent 2>/dev/null || echo 0) +GPU_TEMP=$(cat /sys/class/hwmon/hwmon2/temp1_input 2>/dev/null | awk '{printf "%.1f", $1/1000}') + +cat > /var/lib/node_exporter/textfile_collector/gpu.prom << EOF +# HELP gpu_busy_percent GPU utilization percentage +# TYPE gpu_busy_percent gauge +gpu_busy_percent{gpu="card0"} ${GPU_BUSY} +# HELP gpu_temperature_celsius GPU temperature +# TYPE gpu_temperature_celsius gauge +gpu_temperature_celsius{gpu="card0"} ${GPU_TEMP} +EOF +``` + +Activate with a cron job: +```bash +echo '* * * * * root /usr/local/bin/gpu-metrics.sh' >> /etc/cron.d/gpu-metrics +``` + +Start `node_exporter` with `--collector.textfile.directory=/var/lib/node_exporter/textfile_collector/`. + +### Grafana System Overview Dashboard (GCN 1.0 Compatible) + +A useful System Overview dashboard for older hardware with limited GPU metrics: +- CPU utilization % per core +- RAM usage (total/used/available) +- Disk usage by mount point +- CPU core temperatures (°C) +- NVMe temperature (°C) +- Board temperature (°C) +- Fan speeds (RPM, PWM %) +- Network I/O + +GPU utilization panel: show a text annotation "GPU utilization not supported on AMD GCN 1.0" rather than leaving the panel empty. + +## Related Concepts + +- [[wiki/concepts/prometheus-joules-watts-gotcha]] — related Prometheus/Grafana metric gotcha for power dashboards +- [[wiki/concepts/beszel-monitoring-deployment]] — Beszel as a simpler monitoring layer that doesn't require GPU-specific metric setup +- [[wiki/homelab/_index]] — homelab hardware and Grafana/Prometheus infrastructure + +## Sources + +- [[daily/2026-04-21.md]] — Grafana System Overview dashboard creation; AMD Oland (GCN 1.0) confirmed not to support `gpu_busy_percent` via sysfs; Intel HD 630 similarly limited; textfile collector infrastructure set up for future GPU upgrades; cron activation left pending diff --git a/wiki/concepts/ollama-lxc-ram-requirements.md b/wiki/concepts/ollama-lxc-ram-requirements.md new file mode 100644 index 0000000..fb45c98 --- /dev/null +++ b/wiki/concepts/ollama-lxc-ram-requirements.md @@ -0,0 +1,91 @@ +--- +title: "Ollama in LXC — RAM Requirements and Intel oneAPI Overhead" +aliases: [ollama-lxc, ollama-ram, ollama-proxmox, ollama-cpu-inference] +tags: [ollama, lxc, proxmox, homelab, llm, ram, inference] +sources: + - "daily/2026-04-21.md" +created: 2026-04-21 +updated: 2026-04-21 +--- + +# Ollama in LXC — RAM Requirements and Intel OneAPI Overhead + +Ollama running inside a Proxmox LXC container loads the Intel oneAPI toolkit on startup even when doing CPU-only inference, consuming approximately 1.8 GB of RAM as constant overhead before any model is loaded. This makes RAM sizing for Ollama LXC containers non-obvious — the usable headroom is significantly smaller than the raw allocation. + +## Key Points + +- **Intel oneAPI toolkit consumes ~1.8 GB** at Ollama startup regardless of whether a GPU is present — it's loaded speculatively for hardware detection +- **Minimum practical allocation for a 7B model: 6 GB** — 1.8 GB (oneAPI) + ~1.9 GB (model weights) + KV-cache headroom +- At 6 GB RAM, Ollama 7B runs at approximately **10.3 tok/s on CPU** in LXC (tested on server-grade hardware) +- **4 GB allocation is insufficient** — leaves only ~562 MB free after oneAPI loads, causing OOM errors or severe swapping when loading a 7B model +- GPU passthrough to LXC would eliminate the oneAPI overhead penalty and dramatically increase throughput; requires runtime configuration at the Proxmox level + +## Details + +### Why oneAPI Loads on CPU-Only Inference + +Ollama includes Intel oneAPI libraries for hardware acceleration detection. When Ollama starts, it probes available hardware — including Intel CPU extensions and integrated/discrete GPUs. The oneAPI runtime is initialized during this probe even if no compatible GPU is found and inference ultimately falls back to pure CPU. This is a known overhead, not a bug. + +The practical effect: an LXC with 4 GB RAM appears to have plenty of headroom in `free -h` before Ollama launches, but after startup the node is nearly full before any model inference begins. + +### RAM Sizing Guide + +| Model Size | oneAPI Overhead | Model Weights | Min Allocation | Recommended | +|------------|-----------------|---------------|----------------|-------------| +| 7B (Q4) | ~1.8 GB | ~1.9 GB | 4.5 GB | 6 GB | +| 7B (Q8) | ~1.8 GB | ~3.8 GB | 6.5 GB | 8 GB | +| 13B (Q4) | ~1.8 GB | ~3.5 GB | 6.5 GB | 8 GB | +| 70B (Q4) | ~1.8 GB | ~19 GB | 22 GB | 24 GB | + +Add 1–2 GB beyond minimum for KV-cache during active inference (context accumulation). + +### Diagnosing RAM Exhaustion + +```bash +# Check available RAM before Ollama loads +free -h + +# Check after Ollama starts (before any model pull) +systemctl start ollama +sleep 10 +free -h +# ~1.8 GB should disappear here + +# Check model-loaded footprint +ollama pull llama3 +free -h +``` + +If `ollama pull` hangs or returns OOM errors, the LXC allocation is insufficient even with a model that theoretically fits. + +### Releasing RAM from Unused LXC Containers + +The recommended approach when RAM is needed: shut down unused containers rather than reducing allocations of running services. On this homelab: + +```bash +# Shut down ERPNext (CT100 originally, later renamed) and Monica (CT106) +# to free ~6 GB for Ollama upgrade from 4 GB → 6 GB +ssh pve "pct stop 100 && pct stop 106" +``` + +Shutting down a container immediately releases its RAM — the host sees full benefit without any config changes to other running containers. + +### GPU Passthrough Upgrade Path + +CPU inference at ~10 tok/s is adequate for casual use. To significantly increase throughput, pass through a GPU to the LXC container. This requires: +1. Enable IOMMU on the Proxmox host (see `[[wiki/homelab/_index]]`) +2. Add the GPU device to the LXC configuration +3. Install GPU drivers inside the container +4. Ollama will then automatically use the GPU for inference + +The Intel oneAPI overhead remains on the host even with GPU passthrough, but model inference itself moves to the GPU. + +## Related Concepts + +- [[wiki/concepts/proxmox-vm-management-methods]] — approaches for managing Proxmox VMs and LXC containers +- [[wiki/concepts/homarr-proxmox-integration]] — Proxmox LXC management and container resource monitoring +- [[wiki/homelab/_index]] — full homelab infrastructure including Proxmox setup + +## Sources + +- [[daily/2026-04-21.md]] — Ollama CT100 had only 562 MB free despite 4 GB allocation; root cause identified as Intel oneAPI toolkit; RAM raised to 6 GB by shutting down ERPNext + Monica; Ollama ran at 10.3 tok/s on CPU after fix diff --git a/wiki/concepts/prometheus-joules-watts-gotcha.md b/wiki/concepts/prometheus-joules-watts-gotcha.md new file mode 100644 index 0000000..0eb808d --- /dev/null +++ b/wiki/concepts/prometheus-joules-watts-gotcha.md @@ -0,0 +1,89 @@ +--- +title: "Prometheus — rate(joules_total) Already Returns Watts" +aliases: [prometheus-watts, prometheus-joules, promql-power-metrics, grafana-power-dashboard] +tags: [prometheus, promql, grafana, metrics, power, homelab, monitoring] +sources: + - "daily/2026-04-21.md" +created: 2026-04-21 +updated: 2026-04-21 +--- + +# Prometheus — rate(joules_total) Already Returns Watts + +When calculating power consumption from Prometheus energy counters, `rate(joules_total[interval])` already produces the result in **watts** (joules per second). Multiplying this by 1000 converts watts to milliwatts — not to kilowatts as developers sometimes assume — causing a 1000× display error in Grafana dashboards where values show as kilowatts instead of watts. + +## Key Points + +- **`rate(joules_total[1m])` returns watts** — rate of joules per second = watts (by definition: 1 W = 1 J/s) +- **Do not multiply by 1000** — this gives milliwatts disguised as kilowatts (both wrong units for typical server power readings) +- The bug manifests as: Grafana dashboard shows "2500 W" when the server is actually drawing 2.5 W, or shows "250 W" when actual draw is 0.25 W +- Correct PromQL for watts: `rate(node_cpu_seconds_total{mode!="idle"}[1m])` style or directly `rate(rapl_package_joules_total[1m])` +- Always spot-check power readings against a physical power meter or known consumption baseline — 2500 W is implausible for an idle homelab server + +## Details + +### The Physics + +Power (watts) = Energy (joules) ÷ Time (seconds) + +Prometheus `rate()` computes the per-second rate of a counter, so: + +``` +rate(joules_total[1m]) = Δjoules / Δseconds = watts +``` + +This is mathematically identical to the definition of a watt. No conversion factor is needed. + +### Common Node Exporter Metrics for Power + +```promql +# Package power (Intel RAPL or AMD RAPL-equivalent) — result in watts +rate(node_rapl_package_joules_total[1m]) + +# DRAM power +rate(node_rapl_dram_joules_total[1m]) + +# Total server power (if using a PDU or smart plug exporter) +rate(smart_plug_energy_joules_total[1m]) +``` + +All of these return watts without any multiplication. + +### The Bug Pattern + +```promql +# WRONG — produces milliwatts (labeled as W, appears as kW) +rate(rapl_package_joules_total[1m]) * 1000 + +# CORRECT — watts +rate(rapl_package_joules_total[1m]) +``` + +A developer sees the raw joule values (large counter numbers), assumes `rate()` returns joules/minute, and multiplies by 1000 to "convert to watts". But `rate()` always normalizes to per-second, so the output is already watts. + +### Grafana Dashboard Fix + +In the Grafana panel editor: +1. Remove the `* 1000` from the PromQL expression +2. Verify the unit in **Standard options → Unit** is set to `Watt (W)`, not `kilowatt (kW)` +3. Check the Y-axis maximum — if it was set based on the wrong scale (e.g., 3000 for "3000 W"), adjust to a realistic maximum + +### Sanity Check Values + +Typical homelab server power consumption: +- Idle, low-spec (N100, small board): 8–20 W +- Idle, mid-range (Ryzen 5, full tower): 30–80 W +- Idle, server-grade (Xeon, IPMI): 80–150 W +- Full load (gaming GPU active): 200–400 W + +If Grafana shows a homelab server consuming 2500 W idle, the metric is wrong. + +## Related Concepts + +- [[wiki/concepts/old-gpu-sysfs-metrics]] — related Grafana/Prometheus metrics topic for GPU utilization on older hardware +- [[wiki/concepts/beszel-monitoring-deployment]] — Beszel as a monitoring alternative to Prometheus for simpler homelab setups +- [[wiki/homelab/_index]] — homelab Grafana/Prometheus stack context + +## Sources + +- [[daily/2026-04-21.md]] — Grafana Power & Cost dashboard showing watts as kilowatts; root cause was `* 1000` in PromQL where `rate(joules_total)` already returns watts; fixed by removing the multiplier diff --git a/wiki/concepts/proxmox-container-502-misdiagnosis.md b/wiki/concepts/proxmox-container-502-misdiagnosis.md new file mode 100644 index 0000000..6464b71 --- /dev/null +++ b/wiki/concepts/proxmox-container-502-misdiagnosis.md @@ -0,0 +1,99 @@ +--- +title: "Proxmox — 502 Bad Gateway Does Not Mean a Dead Container" +aliases: [proxmox-502, lxc-502, container-misdiagnosis, proxmox-nginx-502] +tags: [proxmox, lxc, nginx, debugging, homelab, incident, destructive-action] +sources: + - "daily/2026-04-21.md" +created: 2026-04-21 +updated: 2026-04-21 +--- + +# Proxmox — 502 Bad Gateway Does Not Mean a Dead Container + +A 502 Bad Gateway error from an LXC container does not indicate that the container or its primary service is dead. 502 means the reverse proxy (often Nginx) received no valid response from the upstream — the upstream could be a misconfigured hostname, a wrong port, or a crashed sub-process. Deleting a container based solely on a 502 response risks destroying a healthy service. + +## Key Points + +- **502 = upstream unreachable, not container dead** — the container is running; something inside it (Nginx config, service crash, wrong hostname) is causing the 502 +- **Always visually confirm which container the user is pointing at** before any destructive action (delete, restore, wipe) +- The specific incident: CT107 (Homarr, working) showed 502 because Nginx inside CT107 was proxying to hostname `homarr` (unresolvable) instead of `127.0.1.1`; CT103 was the unused/old container — the wrong one was deleted +- **After deleting the wrong container and restoring from backup**, the Nginx config inside needed manual fixing — backups restore data but not post-deploy config fixes applied after the backup was made +- Proxy managers (NPM, Caddy) on a different LXC than the service add an indirection layer — 502 from NPM means NPM can't reach the service, not that the service container itself is broken + +## Details + +### The Incident (2026-04-21) + +**Setup:** +- CT107 = Homarr (the real, working container — had been restored from backup previously) +- CT103 = old/unused Homarr container (should have been the one to delete) + +**What happened:** +1. CT107 returned 502 via NPM +2. Conclusion: "CT107 is the broken/wrong container" → CT107 deleted +3. Reality: CT107's Nginx config was proxying to hostname `homarr` (unresolvable in the container) instead of `127.0.1.1`; fixing the Nginx config would have resolved the 502 +4. CT103 (the unused one) survived the session; CT107 had to be restored from backup + +**Root cause:** 502 ≠ container is dead. 502 meant Nginx inside CT107 was misconfigured. + +### Diagnosing 502 Before Destructive Actions + +```bash +# Step 1: Confirm the container is running +ssh pve "pct status 107" +# Should show: status: running + +# Step 2: Enter the container and check the service directly +ssh pve "pct exec 107 -- systemctl status homarr" +# or +ssh pve "pct exec 107 -- curl -s http://127.0.1.1:7575/api/health" + +# Step 3: Check what Nginx/reverse proxy is configured to proxy to +ssh pve "pct exec 107 -- cat /etc/nginx/sites-enabled/*" +# Look for: proxy_pass http://homarr:7575 ← "homarr" hostname won't resolve + +# Step 4: Check if the hostname resolves inside the container +ssh pve "pct exec 107 -- getent hosts homarr" +# No output = unresolvable hostname = 502 + +# Fix: change proxy_pass to use 127.0.1.1 or 127.0.0.1 +ssh pve "pct exec 107 -- sed -i 's|proxy_pass http://homarr|proxy_pass http://127.0.1.1|g' /etc/nginx/sites-enabled/homarr" +ssh pve "pct exec 107 -- nginx -s reload" +``` + +### Common Causes of 502 in LXC Containers + +| Root Cause | Symptom | Fix | +|-----------|---------|-----| +| Nginx proxying to unresolvable hostname | 502 on all requests | Change `proxy_pass` to use IP | +| Service crashed / not started | 502, service not in `ps aux` | `systemctl restart ` | +| Wrong port in proxy config | 502 even with correct hostname | Fix port number in proxy config | +| ARP cache stale | Intermittent 502 | `ip neigh flush dev eth0` | +| Container networking issue | 502 + ping fails from inside | Check LXC network config | + +### Pre-Destructive-Action Checklist + +Before deleting, wiping, or restoring over any Proxmox container: + +1. **`pct status `** — confirm which VMID you're targeting +2. **Ask user** to point at the container in the Proxmox UI web interface (visual confirmation) +3. **`pct exec -- `** — confirm the service state independently of the proxy layer +4. **Check NPM/Caddy config** — is the 502 from the proxy layer, not the container? +5. Only proceed with destructive action if the container is confirmed unnecessary or truly broken + +### Backup + Restore Nuance + +When a container is restored from backup, it returns to the state at backup time. Any configuration changes made after the backup — including fixing misconfigured Nginx, adding new env vars, or updating service configs — are lost and must be re-applied manually. + +A restored container may therefore show the same 502 that triggered the investigation, because the post-backup Nginx fix was never applied to the backup. + +## Related Concepts + +- [[wiki/concepts/lxc-arp-cache-api-failures]] — another LXC failure mode that appears as API/connectivity errors but has a non-obvious cause +- [[wiki/concepts/homarr-proxmox-integration]] — Homarr-specific configuration (the service involved in this incident) +- [[wiki/connections/lxc-networking-api-failures]] — broader pattern of LXC failures with misleading symptoms +- [[wiki/homelab/_index]] — Proxmox and LXC infrastructure context + +## Sources + +- [[daily/2026-04-21.md]] — Critical incident: CT107 (working Homarr) deleted based on 502; root cause was Nginx inside CT107 proxying to hostname `homarr` instead of `127.0.1.1`; CT103 was the unused container; CT107 restored from backup; Nginx config fixed post-restore; lesson: 502 ≠ wrong container, always confirm visually before destructive actions diff --git a/wiki/concepts/uptime-kuma-socketio-management.md b/wiki/concepts/uptime-kuma-socketio-management.md new file mode 100644 index 0000000..f233773 --- /dev/null +++ b/wiki/concepts/uptime-kuma-socketio-management.md @@ -0,0 +1,120 @@ +--- +title: "Uptime Kuma — No REST API; Socket.IO and SQLite Management" +aliases: [uptime-kuma-api, uptime-kuma-management, uptime-kuma-sqlite, uptime-kuma-socketio] +tags: [uptime-kuma, monitoring, sqlite, socketio, homelab, selfhosted] +sources: + - "daily/2026-04-21.md" +created: 2026-04-21 +updated: 2026-04-21 +--- + +# Uptime Kuma — No REST API; Socket.IO and SQLite Management + +Uptime Kuma does not expose a traditional REST API for programmatic management. All UI interactions use Socket.IO (WebSocket). For automated or scripted operations — like deleting monitors in bulk — the most practical approach is direct SQLite access to the database file at `/opt/uptime-kuma/data/kuma.db`. + +## Key Points + +- **No REST API** — Uptime Kuma uses Socket.IO exclusively; there are no `GET /monitors` or `DELETE /monitors/123` endpoints +- **SQLite database** at `/opt/uptime-kuma/data/kuma.db` is the authoritative data store; safe to query and modify when Kuma is stopped +- **Delete monitors via SQL:** `DELETE FROM monitor WHERE id IN (6, 10)` — simpler than scripting Socket.IO +- **Monitor IDs are visible in the browser** — open Uptime Kuma UI, click a monitor, the ID appears in the URL `/#/details/6` +- The unofficial community API (`louislam/uptime-kuma-api` npm package) wraps Socket.IO and is the safest programmatic interface + +## Details + +### Why There Is No REST API + +Uptime Kuma's architecture was originally designed as a real-time dashboard — all state changes propagate via WebSocket events to connected clients. The author chose Socket.IO as the sole communication layer. This works well for the UI but makes scripted management non-standard. + +The official project position (as of 2026-04): REST API is a feature request, not planned for the near term. Use the Socket.IO interface or direct SQLite access. + +### Finding Monitor IDs + +**Via the UI:** +1. Open Uptime Kuma at `http://:3001` +2. Click a monitor +3. URL changes to `http://:3001/#/details/6` — the ID is `6` + +**Via SQLite:** +```bash +sqlite3 /opt/uptime-kuma/data/kuma.db \ + "SELECT id, name, url FROM monitor ORDER BY id;" +``` + +### Deleting Monitors via SQLite + +When running Uptime Kuma in a Proxmox LXC container: + +```bash +# Access the container and delete monitors +ssh pve "pct exec 110 -- sqlite3 /opt/uptime-kuma/data/kuma.db \ + 'DELETE FROM monitor WHERE id IN (6, 10)'" +``` + +Or if logged into the LXC directly: +```bash +sqlite3 /opt/uptime-kuma/data/kuma.db \ + "DELETE FROM monitor WHERE id IN (6, 10);" +``` + +Uptime Kuma can remain running during read queries but **stop it before writes** to avoid database corruption: + +```bash +# Safer approach for modifications +systemctl stop uptime-kuma +sqlite3 /opt/uptime-kuma/data/kuma.db "DELETE FROM monitor WHERE id IN (6, 10);" +systemctl start uptime-kuma +``` + +### Community Socket.IO API (Programmatic Use) + +For more complex automation, the `louislam/uptime-kuma-api` npm package (unofficial) wraps the Socket.IO interface: + +```js +const { UptimeKumaApi } = require('uptime-kuma-api'); + +const api = new UptimeKumaApi({ baseURL: 'http://192.168.1.x:3001' }); +await api.loginByPassword({ username: 'admin', password: '...' }); +await api.deleteMonitor(6); +await api.disconnect(); +``` + +This is more reliable than raw Socket.IO scripting and survives minor Kuma version updates. + +### Key Database Tables + +```sql +-- Main monitor config +SELECT id, name, type, url, active FROM monitor; + +-- Notification channels +SELECT id, name, type FROM notification; + +-- Monitor-to-notification mapping +SELECT monitor_id, notification_id FROM monitor_notification; + +-- Heartbeat history (can grow large) +SELECT monitor_id, status, time FROM heartbeat ORDER BY time DESC LIMIT 20; +``` + +### Connecting Uptime Kuma Alerts to Webhook + +To send alerts to a monitoring agent webhook (e.g., at `http://192.168.1.225:9111/alert/uptime-kuma`): + +1. **Settings → Notifications → Add Notification** +2. Type: `Webhook` +3. URL: `http://192.168.1.225:9111/alert/uptime-kuma` +4. Save +5. For each monitor: open monitor → Edit → Notifications → enable the webhook + +This must be done via the UI — there is no bulk "assign notification to all monitors" operation. + +## Related Concepts + +- [[wiki/concepts/beszel-monitoring-deployment]] — Beszel as a complementary monitoring tool (server metrics vs uptime monitoring) +- [[wiki/concepts/homarr-proxmox-integration]] — Homarr integrates with Uptime Kuma to display monitor status in the dashboard +- [[wiki/homelab/_index]] — full homelab monitoring stack context + +## Sources + +- [[daily/2026-04-21.md]] — ERPNext (monitor ID 6) and Monica (ID 10) decommissioned; deletion via SQLite recommended as `pct exec 110 -- sqlite3 ... 'DELETE FROM monitor WHERE id IN (6, 10)'`; Uptime Kuma uses Socket.IO not REST API confirmed diff --git a/wiki/infrastructure/_index.md b/wiki/infrastructure/_index.md new file mode 100644 index 0000000..7bba497 --- /dev/null +++ b/wiki/infrastructure/_index.md @@ -0,0 +1,43 @@ +--- +tags: [infrastructure, index] +updated: 2026-04-24 +--- + +# Infrastructure — Index + +Server inventory for all SSH-accessible machines. Audited 2026-04-24. + +## Oliver Agency Servers (GCP) + +| Article | Server | IP | Role | +|---------|--------|----|------| +| [[wiki/infrastructure/server-optical\|server-optical]] | optical-web-1 | 10.220.168.5 | Main AI prod — 35+ apps, systemd | +| [[wiki/infrastructure/server-optical-dev\|server-optical-dev]] | optical-dev | 10.220.168.9 | Docker staging — ppt-tool, cc-dashboard, semblance, 15+ apps | +| [[wiki/infrastructure/server-optical-prod\|server-optical-prod]] | optical-prod | 10.220.168.8 | Minimal / secondary prod | +| [[wiki/infrastructure/server-librechat\|server-librechat]] | librechat-dev + prod | 10.220.168.2 / .4 | LibreChat AI chat platform (both envs) | +| [[wiki/infrastructure/server-modocmms\|server-modocmms]] | modcomms-01 | 10.220.168.6 | ModoCMMS staging + prod (Apache) | +| [[wiki/infrastructure/server-baic\|server-baic]] | web-03 | 10.220.72.13 | Main web host — 40+ domains, oliver.agency | +| [[wiki/infrastructure/server-box-cli\|server-box-cli]] | box-cli-01 | 10.220.176.3 | Ford/L'Oréal hotfolder, CentOS 7, 1TB NFS | + +## Personal / Aimpress + +| Article | Server | IP | Role | +|---------|--------|----|------| +| [[wiki/infrastructure/server-aimpress\|server-aimpress]] | c2-15-uk1 | 57.128.160.249 | Aimpress VPS — Mailcow, n8n, Traefik | +| [[wiki/infrastructure/server-pve\|server-pve]] | pve | 192.168.1.48 | Proxmox homelab — 8 containers + Kali VM | + +## Quick Reference + +| Article | Purpose | +|---------|---------| +| [[wiki/infrastructure/ssh-aliases\|ssh-aliases]] | All aliases, IPs, keys, health-check one-liner | + +## ⚠ Known Issues (as of 2026-04-24) + +- `box-cli` — CentOS 7 (EOL since Jun 2024) — needs OS migration +- `librechat-prod` — disk 65% used on 484 GB — monitor backups +- `librechat-prod` — MongoDB :27017 listening on all interfaces — verify firewall +- `optical-dev` — hp-prod-tracker + dow-prod-tracker containers **unhealthy** +- `pve` — local-lvm pool at 71% — monitor and clean up +- `pve` — Uptime Kuma webhook to monitoring-agent not yet configured +- `baic` (Grafana) — default admin:admin password not changed yet diff --git a/wiki/infrastructure/server-aimpress.md b/wiki/infrastructure/server-aimpress.md new file mode 100644 index 0000000..3327294 --- /dev/null +++ b/wiki/infrastructure/server-aimpress.md @@ -0,0 +1,121 @@ +--- +tags: [infrastructure, server, vps, aimpress, personal] +updated: 2026-04-24 +--- + +# aimpress — Aimpress LTD Personal VPS + +> SSH alias: `aimpress` → `ubuntu@57.128.160.249:1220` +> Key: `~/.ssh/id_rsa` + +## Overview + +Personal VPS for Aimpress LTD (Vadym's company). Hosts personal/business infrastructure: email (Mailcow), CRM (Twenty), booking (Cal.com), AI chatbot, Rocket.Chat, and automation (n8n). Managed via Traefik reverse proxy. All apps run as Docker Compose stacks. + +- **Hostname**: c2-15-uk1 +- **Provider**: OVHcloud (UK, likely London) +- **OS**: Ubuntu 24.04.4 LTS +- **IP**: 57.128.160.249 (public) +- **SSH Port**: 1220 (non-standard) +- **Disk**: 96 GB / 49 GB used (51%) +- **Reverse proxy**: Traefik (Docker container, ports 80/443) +- **Domain**: ai-impress.com + +## Docker Containers Running + +### Email — Mailcow + +| Container | Image | Purpose | +|-----------|-------|---------| +| mailcowdockerized-nginx-mailcow-1 | ghcr.io/mailcow/nginx:1.06 | Mailcow web UI | +| mailcowdockerized-postfix-mailcow-1 | ghcr.io/mailcow/postfix | SMTP (25/465/587) | +| mailcowdockerized-dovecot-mailcow-1 | ghcr.io/mailcow/dovecot | IMAP/POP3 (110/143/993/995) | +| mailcowdockerized-rspamd-mailcow-1 | spam filter | — | +| mailcowdockerized-clamd-mailcow-1 | ClamAV | Antivirus | +| mailcowdockerized-mysql-mailcow-1 | mariadb:10.11 | Mailcow DB (127.0.0.1:13306) | +| mailcowdockerized-redis-mailcow-1 | redis:7.4 | Cache (127.0.0.1:7654) | +| mailcowdockerized-sogo-mailcow-1 | SOGo | Webmail + CalDAV/CardDAV | +| mailcowdockerized-unbound-mailcow-1 | Unbound | DNS resolver | + +### CRM — Twenty CRM + +| Container | Purpose | +|-----------|---------| +| twenty-server | Twenty CRM server | +| twenty-worker | Background jobs | +| twenty-redis | Cache | +| postgres-main | Shared Postgres (5432) | + +### Business Apps + +| Container | Image | Purpose | +|-----------|-------|---------| +| aimpress-calcom | calcom/cal.com:latest | Booking / calendar | +| aimpress-rocketchat | rocket.chat:8.2.0 | Team chat | +| rocketchat-mongo | mongo | Rocket.Chat DB | +| aimpress-chatbot-api | website-chatbot-api | AI chatbot for website | +| aimpress-email-api | website-email-api | Email send API | +| aimpress-chatbot-redis | redis | Chatbot cache | +| aimpress-website | — | Aimpress website | +| n8n | — | Workflow automation | +| vaultwarden | — | Bitwarden-compatible password manager | +| documenso | — | Document signing (like DocuSign) | + +### Side Projects + +| Container | Purpose | +|-----------|---------| +| axil-app-1 | Axil app (port 3000) | +| axil-db-1 | Axil Postgres | +| bookkeeper-agent-bookkeeper-1 | AI bookkeeping agent (port 8000, via proxy :8001) | + +## Nginx Routing + +| Domain | Backend | +|--------|---------| +| axil.ai-impress.com | :3000 | +| bookkeeper.ai-impress.com | :8001 | + +Mailcow uses its own nginx container. Traefik handles routing for most other apps via Docker labels. + +## Ports (External) + +| Port | Service | +|------|---------| +| 25 | SMTP (Postfix) | +| 80 | Traefik HTTP | +| 110 | POP3 (Dovecot) | +| 143 | IMAP (Dovecot) | +| 443 | Traefik HTTPS | +| 465 | SMTPS (Postfix) | +| 587 | SMTP submission (Postfix) | +| 993 | IMAPS (Dovecot) | +| 995 | POP3S (Dovecot) | +| 1220 | SSH (non-standard) | +| 4190 | Sieve (mail filtering) | + +## /opt/ Structure + +Organized by category: +- `00-infrastructure/` — core infra (Traefik, etc.) +- `01-security/` — Vaultwarden, Fail2ban config +- `02-core/` — Mailcow, Twenty CRM +- `03-business/` — Cal.com, Rocket.Chat, chatbot, n8n +- `04-monitoring/` — monitoring stack + +## Key Takeaways + +- **This is personal/Aimpress LTD infrastructure — not Oliver Agency** +- Traefik handles SSL (Let's Encrypt auto-renewal) for all apps +- SSH on port 1220 (not 22) — always specify in commands +- Mailcow is full email stack (SMTP/IMAP/webmail/spam/antivirus) +- linkedin-autopost.service running as systemd (LinkedIn automation) +- qemu-guest-agent suggests this VPS may itself be a VM +- `~/bookkeeper-agent/` — AI bookkeeping agent project (see wiki) +- Password manager (Vaultwarden) self-hosted here + +## Related + +- [[wiki/infrastructure/ssh-aliases|ssh-aliases]] +- [[wiki/infrastructure/server-pve|server-pve]] +- [[wiki/05 Aimpress LTD|Aimpress LTD]] diff --git a/wiki/infrastructure/server-baic.md b/wiki/infrastructure/server-baic.md new file mode 100644 index 0000000..feca970 --- /dev/null +++ b/wiki/infrastructure/server-baic.md @@ -0,0 +1,109 @@ +--- +tags: [infrastructure, server, gcp, baic, web-hosting, oliver-internal] +updated: 2026-04-24 +--- + +# baic — Main Oliver Web Hosting Server (web-03) + +> SSH alias: `baic` → `vadym.samoilenko@10.220.72.13:22` +> Key: `~/.ssh/id_rsa_vadym` + +## Overview + +Despite the `baic` alias, this is actually the **main Oliver Agency web hosting server** — it hosts 40+ domains including all regional oliver.agency sites, ustudio, client preview sites, and internal tools. Also runs ModoCMMS production via Docker. + +- **Hostname**: web-03 +- **Platform**: GCP +- **OS**: Ubuntu 22.04.5 LTS +- **IP**: 10.220.72.13 +- **Web server**: Apache 2.4 +- **Disk**: 582 GB / 337 GB used (58%) + +## Docker Containers + +| Container | Image | Port | Status | +|-----------|-------|------|--------| +| modcomms-prod-backend-1 | modcomms-prod-backend | 0.0.0.0:8000 | Up 22h (healthy) | +| modcomms-prod-postgres-1 | postgres:16-alpine | 0.0.0.0:5432 | Up 22h (healthy) | + +## Hosted Domains (Apache vhosts, 40+) + +### Client / Product Sites + +| Domain | Notes | +|--------|-------| +| baic.oliver.solutions | BAIC client tool | +| barclaycard.oliver.agency | Barclays | +| barclaysxapple.oliver.solutions | Barclays × Apple | +| barclaysxapple-stg.oliver.solutions | Staging | +| oliverai-chat.oliver.solutions | OliverAI chat | +| oxygen.oliver.solutions | Oxygen tool | +| ai-portal.oliver.solutions | AI Portal | +| box-uploader.oliver.solutions | Box uploader | +| marriott-btg.oliver.solutions | Marriott | +| marriottbmd.com | Marriott BMD | +| adjustyourset.com | AdjustYourSet | +| lmaudit.oliver.digital | LM Audit | +| intranet.insideideas.agency | InsideIdeas intranet | +| jakarta.ustudio.global | UStudio Jakarta | +| ustudiostaging.oliver.solutions | UStudio staging | +| ustudiostaging2.oliver.solutions | UStudio staging 2 | +| www.brandtech.plus | BrandTech+ | + +### Oliver Regional Sites + +| Domain | Region | +|--------|--------| +| www.oliver.agency | Main | +| ae.oliver.agency | UAE | +| de.oliver.agency | Germany | +| es.oliver.agency | Spain | +| fr.oliver.agency | France | +| ie.oliver.agency | Ireland | +| nl.oliver.agency | Netherlands | +| sw.oliver.agency | Sweden | +| tr.oliver.agency | Turkey | +| oliverstaging.oliver.agency | Staging | + +### Staging / Preview + +| Domain | Notes | +|--------|-------| +| ae.staging.oliver.digital | UAE staging | +| de.staging.oliver.digital | DE staging | +| es.staging.oliver.digital | ES staging | +| fr.staging.oliver.digital | FR staging | +| nl.staging.oliver.digital | NL staging | +| sw.staging.oliver.digital | SE staging | +| tr.staging.oliver.digital | TR staging | +| oliverweb.staging.oliver.digital | Oliver web staging | +| oliverwebie.staging.oliver.digital | IE staging | +| oliverwebiev2.staging.oliver.digital | IE v2 staging | +| axa1.preview.oliver.digital | AXA preview | +| ays.preview.oliver.digital | AYS preview | + +## Ports + +| Port | Service | +|------|---------| +| 80 / 443 | Apache (40+ vhosts) | +| 5001 | Internal API | +| 5432 | PostgreSQL (ModoCMMS prod) | +| 5666 | NRPE (Nagios Remote Plugin Executor) | +| 8000 | ModoCMMS prod backend (Docker) | + +## Key Takeaways + +- **This is the main web server** — hosts oliver.agency, regional sites, and client previews +- ModoCMMS PRODUCTION runs here as Docker Compose (backend :8000, postgres :5432) +- Named `baic` in SSH config after the BAIC project — but now hosts much more +- CrowdStrike Falcon (via amazon-ssm-agent) for security monitoring +- NFS mount: `172.22.113.5:/prod-filestore` → `/data` (1TB) for file storage +- 40+ Apache vhosts — add new sites via `/etc/apache2/sites-available/` + `a2ensite` +- NRPE on :5666 — Nagios monitoring integration + +## Related + +- [[wiki/infrastructure/ssh-aliases|ssh-aliases]] +- [[wiki/infrastructure/server-modocmms|server-modocmms]] +- [[wiki/client-knowledge/_index|client-knowledge]] diff --git a/wiki/infrastructure/server-box-cli.md b/wiki/infrastructure/server-box-cli.md new file mode 100644 index 0000000..0f8b399 --- /dev/null +++ b/wiki/infrastructure/server-box-cli.md @@ -0,0 +1,74 @@ +--- +tags: [infrastructure, server, gcp, ford, loreal, file-processing] +updated: 2026-04-24 +--- + +# box-cli — Ford / L'Oréal File Processing Server + +> SSH alias: `box-cli` → `vadym.samoilenko@10.220.176.3:22` +> Key: `~/.ssh/id_rsa_vadym` + +## Overview + +Dedicated server for hotfolder-based file processing workflows. Handles Ford QC Box integration and L'Oréal deliverables syncing. Older CentOS 7 server with NFS-mounted production filestore. + +- **Hostname**: box-cli-01 +- **Platform**: GCP +- **OS**: CentOS Linux 7 (Core) — ⚠ EOL since June 2024 +- **IP**: 10.220.176.3 +- **Web server**: None (no Apache/Nginx) +- **Disk**: + - `/dev/sda2`: 128 GB / 79 GB used (62%) + - `/data` (NFS): 1 TB / 502 GB used (49%) — `172.22.113.5:/prod-filestore` + +## Running Services (Key) + +| Service | Purpose | +|---------|---------| +| ford-asset-pack-report.service | Ford Asset Pack SFTP upload reporting | +| ford-asset-pack-sftp.service | Ford Asset Pack SFTP transfer daemon | +| ford-qc-hotfolder-PROD.service | Ford QC Box hotfolder (PROD) | +| ford-qc-hotfolder.service | Ford QC Box hotfolder (DEV/staging) | +| loreal-deliverables.service | L'Oréal deliverables Airtable sync | +| json-processor.service | JSON workflow processor | +| webmin.service | Webmin admin panel | +| xrdp.service | Remote desktop (RDP) | +| fail2ban.service | Brute-force protection | +| zerotier-one.service | ZeroTier VPN overlay | +| falcon-sensor.service | CrowdStrike Falcon EDR | +| libvirtd.service | KVM virtualization (libvirt) | + +## /opt/ Contents + +| Dir | Purpose | +|-----|---------| +| chef | Chef configuration management | +| CrowdStrike | Falcon sensor | +| containerd | Container runtime | + +## Key Architecture + +The server watches specific "hotfolder" directories on the NFS mount (`/data`). When files appear: +1. **ford-qc-hotfolder** — picks up assets, applies QC checks via Box API, uploads approved files +2. **ford-asset-pack-sftp** — transfers asset packs to SFTP endpoints +3. **loreal-deliverables** — syncs deliverable metadata to Airtable + +## Ports + +No public HTTP — only SSH (22) and ZeroTier VPN. + +## Key Takeaways + +- **CentOS 7 is EOL** — migration to RHEL 8/9 or Rocky Linux should be planned +- NFS `/data` is 49% full on 1TB — file retention policy important +- `box-cli` alias name comes from Box.com integration (not the CLI tool) +- Ford QC has TWO services: one for PROD (`ford-qc-hotfolder-PROD`) and one for DEV +- Webmin on port 10000 (internal), xrdp for remote desktop access +- CrowdStrike Falcon monitored by EDR +- libvirtd suggests possible local VMs running (check with `virsh list`) + +## Related + +- [[wiki/infrastructure/ssh-aliases|ssh-aliases]] +- [[wiki/client-knowledge/_index|client-knowledge]] +- [[wiki/tech-patterns/_index|tech-patterns]] diff --git a/wiki/infrastructure/server-librechat.md b/wiki/infrastructure/server-librechat.md new file mode 100644 index 0000000..e18e5db --- /dev/null +++ b/wiki/infrastructure/server-librechat.md @@ -0,0 +1,104 @@ +--- +tags: [infrastructure, server, gcp, librechat, oliver-internal] +updated: 2026-04-24 +--- + +# LibreChat Servers (Dev + Prod) + +## librechat-dev + +> SSH alias: `librechat-dev` → `vadym.samoilenko@10.220.168.2:22` +> Key: `~/.ssh/id_rsa_vadym` + +- **Hostname**: optical-librechat-dev.europe-west2-b.c.optical-414516.internal +- **Platform**: GCP europe-west2-b +- **OS**: Ubuntu 24.04 LTS +- **Disk**: 116 GB / ~58 GB used (50%) + +### Apps in `/opt/` + +| Dir | Purpose | +|-----|---------| +| LibreChat | LibreChat app (Node.js/React) | +| LibreCodeInterpreter | Code execution sandbox for LibreChat | +| librechat-analytics | Analytics pipeline | +| librechat-balances | Token balance tracking | +| site24x7 | Monitoring agent | + +### Ports + +| Port | Service | +|------|---------| +| 80 / 443 | — (SSL, no nginx sites-enabled found — likely nginx.conf inline or traefik) | +| 3080 | LibreChat main app | +| 3001 / 3002 | LibreChat workers/API | +| 6379 | Redis | +| 6791 | Internal | +| 8000 | Analytics / balance API | +| 8888 | Jupyter / Code Interpreter | +| 9000 / 9001 | Code Interpreter | + +### Key Services (systemd) + +- elastic-collector.service, filebeat.service, metricbeat.service — log/metric shipping to Elasticsearch +- site24x7monagent.service — uptime monitoring +- docker.service — Docker engine (no containers shown but containerd running) + +--- + +## librechat-prod + +> SSH alias: `librechat-prod` → `vadym.samoilenko@10.220.168.4:22` +> Key: `~/.ssh/id_rsa_vadym` + +- **Hostname**: optical-librechat.europe-west2-c.c.optical-414516.internal +- **Platform**: GCP europe-west2-c +- **OS**: Ubuntu 24.04 LTS +- **Disk**: 484 GB / 313 GB used (65%) — **⚠ getting full** + +### Apps in `/opt/` + +| Dir | Purpose | +|-----|---------| +| LibreChat | LibreChat production app | +| LibreCodeInterpreter | Code execution sandbox | +| Librechat-backups | Backup storage | +| agent-sync | Agent synchronization service | +| librechat-analytics | Analytics | +| librechat-balances | Token balance tracking | + +### Ports + +| Port | Service | +|------|---------| +| 443 | HTTPS (SSL) | +| 3080 | LibreChat app | +| 3001 / 3002 | LibreChat workers | +| 5001 | mongodb-api (custom API on top of MongoDB) | +| 6379 | Redis | +| 8000 | Internal API | +| 9000 / 9001 | Code Interpreter | +| 20201/20202 | Monitoring agents | +| 27017 | MongoDB (⚠ exposed publicly — confirm firewall rules) | + +### Key Services (systemd) + +- **mongodb-api.service** — custom MongoDB REST API +- google-cloud-ops-agent — GCP metrics/logging +- filebeat, metricbeat — Elastic log shipping +- site24x7monagent — uptime monitoring + +--- + +## Key Takeaways + +- librechat-prod disk is **65% full** on a 484 GB disk (313 GB used) — monitor backups +- MongoDB :27017 on prod is listening on all interfaces — verify GCP firewall +- Both servers use SSL but no nginx sites-enabled found — likely nginx.conf inline or cert-manager +- LibreCodeInterpreter provides sandboxed code execution (port 8888 Jupyter + 9000/9001) +- Many developers have home dirs on both servers (team-wide access) + +## Related + +- [[wiki/infrastructure/ssh-aliases|ssh-aliases]] +- [[wiki/infrastructure/server-optical|server-optical]] diff --git a/wiki/infrastructure/server-modocmms.md b/wiki/infrastructure/server-modocmms.md new file mode 100644 index 0000000..0ca4529 --- /dev/null +++ b/wiki/infrastructure/server-modocmms.md @@ -0,0 +1,65 @@ +--- +tags: [infrastructure, server, gcp, modocmms, oliver-internal] +updated: 2026-04-24 +--- + +# modocmms-dev — ModoCMMS Application Server + +> SSH alias: `modocmms-dev` → `vadym.samoilenko@10.220.168.6:22` +> Key: `~/.ssh/id_rsa_vadym` + +## Overview + +Dedicated server for the ModoCMMS product (maintenance management system). Runs two environments: staging and production, both on the same machine via Apache vhosts. + +- **Hostname**: modcomms-01.europe-west2-c.c.optical-414516.internal +- **Platform**: GCP europe-west2-c +- **OS**: Ubuntu 24.04.3 LTS +- **IP**: 10.220.168.6 +- **Web server**: Apache 2.4 +- **Disk**: 96 GB / ~12 GB used (12%) — plenty of space + +## Applications in `/opt/` + +| Dir | Environment | +|-----|-------------| +| `/opt/modcomms-dev/` | Staging / dev version | +| `/opt/modcomms-prod/` | Production version | + +## Apache Vhosts + +### modcomms-staging.oliver.solutions (staging) + +- **Document root**: `/var/vhosts/modcomms-staging.oliver.solutions/htdocs/` +- **Backend API**: `ProxyPass /back/ http://localhost:8001/` +- **WebSocket**: `ProxyPass /back/ws/analyze ws://localhost:8001/ws/analyze` +- **Logs**: `/var/vhosts/modcomms-staging.oliver.solutions/logs/` +- HTTPS redirect enforced (X-Forwarded-Proto) + +### modcomms.oliver.solutions (production) + +- **Document root**: `/var/vhosts/modcomms.oliver.solutions/htdocs/` +- Backend proxied similarly to staging + +## Ports + +| Port | Service | +|------|---------| +| 80 / 443 | Apache | +| 5432 | PostgreSQL | +| 8001 | ModoCMMS backend (FastAPI/Python) | +| 20201 / 20202 | Monitoring | + +## Key Takeaways + +- Two envs on one box: staging at modcomms-staging.oliver.solutions, prod at modcomms.oliver.solutions +- Backend runs on :8001 (Python), frontend served as static files from Apache htdocs +- WebSocket support for `/back/ws/analyze` — ML analysis endpoint +- PostgreSQL on :5432 (publicly exposed — verify firewall) +- No Docker — pure systemd/Apache deployment +- Site24x7 and google-cloud-ops-agent provide monitoring + +## Related + +- [[wiki/infrastructure/ssh-aliases|ssh-aliases]] +- [[wiki/tech-patterns/_index|tech-patterns]] diff --git a/wiki/infrastructure/server-optical-dev.md b/wiki/infrastructure/server-optical-dev.md new file mode 100644 index 0000000..f710722 --- /dev/null +++ b/wiki/infrastructure/server-optical-dev.md @@ -0,0 +1,122 @@ +--- +tags: [infrastructure, server, gcp, oliver-internal] +updated: 2026-04-24 +--- + +# optical-dev — Docker App Dev/Staging Server + +> SSH alias: `optical-dev` → `vadym.samoilenko@10.220.168.9:22` +> Key: `~/.ssh/id_rsa_vadym` + +## Overview + +Primary Docker-based development and staging server for Oliver internal tools. All apps run as Docker Compose stacks in `/opt/{name}/`. Apache handles routing on port 80 (SSL terminated at upstream load balancer). + +- **Hostname**: (GCP instance, europe-west2) +- **Platform**: GCP +- **OS**: Ubuntu (likely 24.04) +- **IP**: 10.220.168.9 +- **Web server**: Apache 2.4 (single vhost: `optical-dev.oliver.solutions`) +- **Domain**: optical-dev.oliver.solutions +- **Disk**: 532 GB / 288 GB used (55%) + +## Docker Containers Running + +| Container | Image | Exposed Port | Status | +|-----------|-------|-------------|--------| +| cc-dashboard-app-1 | cc-dashboard-app | 8800 | Up 4 weeks | +| cc-dashboard-db-1 | postgres:16-alpine | — | Up 4 weeks | +| ppt-tool-web-1 | ppt-tool-web | 127.0.0.1:3000 | Up 3 weeks | +| ppt-tool-api-1 | ppt-tool-api | 127.0.0.1:8001 | Up 3 weeks | +| ppt-tool-worker-1 | ppt-tool-worker | — | Up 3 weeks | +| ppt-tool-postgres-1 | postgres:16-alpine | — | Up 4 weeks | +| ppt-tool-redis-1 | redis:7-alpine | — | Up 4 weeks | +| semblance-backend-1 | semblance-backend | 127.0.0.1:5137 | Up 4 weeks | +| semblance-mongo-1 | mongo:7 | 127.0.0.1:27017 | Up 4 weeks | +| olivas-backend-1 | olivas-backend | 0.0.0.0:8000 | Up 4 weeks | +| olivas-postgres-1 | postgres:16-alpine | — | Up 4 weeks | +| barclays-banner-builder-api-1 | barclays-banner-builder-api | 127.0.0.1:8010 | Up 3 days | +| barclays-banner-builder-worker-1 | — | — | Up 3 days | +| barclays-banner-builder-redis-1 | redis:7-alpine | — | Up 6 days | +| barclays-banner-builder-postgres-1 | pgvector/pgvector:pg16 | — | Up 6 days | +| social-mi-bi-api-1 | social-mi-bi-api | 127.0.0.1:8302 | Up 3 days | +| social-mi-bi-worker-1 | — | — | Up 3 days | +| social-mi-bi-minio-1 | minio/minio | 127.0.0.1:8303→9000 | Up 3 days | +| social-mi-bi-db-1 | postgres:16-alpine | — | Up 3 days | +| social-mi-bi-redis-1 | redis:7-alpine | — | Up 3 days | +| social-reporting-social-listening-1 | social-reporting | 127.0.0.1:3456 | Up 8 days | +| social-reporting-db-1 | postgres:16-alpine | 0.0.0.0:5436 | Up 8 days | +| amazon-transcreation-frontend-1 | — | 127.0.0.1:3050 | Up 7 days | +| amazon-transcreation-backend-1 | — | 127.0.0.1:8040 | Up 7 days | +| amazon-transcreation-celery_worker-1 | — | — | Up 7 days | +| amazon-transcreation-db-1 | postgres:16 | 127.0.0.1:5492 | Up 7 days | +| amazon-transcreation-redis-1 | redis:7-alpine | 127.0.0.1:6389 | Up 7 days | +| gmal-scope-builder-backend-1 | — | 127.0.0.1:8002 | Up 10 days | +| gmal-scope-builder-db-1 | postgres:16-alpine | — | Up 3 weeks | +| salary-benchmark-app | salary-benchmark-app | 127.0.0.1:8100 | Up 3 days | +| salary-benchmark-db | postgres:16-alpine | — | Up 3 days | +| hp-prod-tracker-app-1 | hp-prod-tracker-app | 0.0.0.0:3001 | Up 7 days (unhealthy) | +| hp-prod-tracker-db-1 | pgvector/pgvector:pg17 | 0.0.0.0:5491 | Up 8 days | +| dow-prod-tracker-app-1 | dow-prod-tracker-app | 0.0.0.0:3002 | Up 41 hours (unhealthy) | +| dow-prod-tracker-db-1 | pgvector/pgvector:pg17 | 0.0.0.0:5493 | Up 41 hours | +| hp-studios-ai-content-agent-frontend-1 | — | 127.0.0.1:22970 | Up 37 hours | +| hp-studios-ai-content-agent-api-1 | — | 127.0.0.1:22409 | Up 37 hours | +| hp-studios-ai-content-agent-worker-1 | — | — | Up 37 hours | +| hp-studios-ai-content-agent-redis-1 | redis:7-alpine | — | Up 37 hours | +| hp-studios-ai-content-agent-postgres-1 | pgvector/pgvector:pg16 | — | Up 37 hours | + +## Apache Routing (optical-dev.oliver.solutions) + +SSL terminated upstream. Single vhost on `:80`. + +| Path | Backend | App | +|------|---------|-----| +| `/ppt-tool/api/v1/` | :8001 | DeckForge FastAPI | +| `/ppt-tool` | :3000 | DeckForge Next.js | +| `/api/` | :8000 | OliVAS FastAPI | +| `/olivas/` | static `/var/www/html/olivas` | OliVAS SPA | +| `/semblance_back/` | :5137 | Semblance Quart backend (WebSocket) | +| `/semblance/` | static `/var/www/html/semblance` | Semblance SPA | +| `/cc-dashboard/api/` | :8800 | CC Dashboard FastAPI | +| `/cc-dashboard/` | static `/var/www/html/cc-dashboard` | CC Dashboard SPA | +| `/gsb/api/` | :8002 | GMAL Scope Builder API | +| `/gsb/` | static `/var/www/html/gmal-scope-builder` | GMAL SPA | +| `/hp-prod-tracker/` | :3001 | HP Prod Tracker | +| `/amazon-transcreation/` | :8040 / :3050 | via Include | +| `/barclays*` | :8010 | Barclays Banner Builder via Include | +| `/salary-benchmark*` | :8100 | via Include | +| `/social-mi-bi*` | :8302 | via Include | +| `/dow-prod-tracker*` | :3002 | via Include | +| `/ai-qc*` | — | via Include | +| `/hp-content-agent*` | :22409 / :22970 | via Include | + +Each app defines its own Apache include at `/opt/{name}/deploy/apache-*.conf`. + +## Deployment Pattern + +```bash +# Standard deploy +cd /opt/{app-name} +git pull +docker compose -f docker-compose.prod.yml build +docker compose -f docker-compose.prod.yml up -d +# or +docker compose up -d --build +``` + +Static SPA builds go to `/var/www/html/{app-name}/`. + +## Key Takeaways + +- All apps in `/opt/{name}/` as Docker Compose stacks +- Apache config: `/etc/apache2/sites-enabled/optical-dev.oliver.solutions.conf` +- Per-app Apache includes let each project own its routing config +- `hp-prod-tracker` and `dow-prod-tracker` currently **unhealthy** — check logs +- `deploy-api` service is running (uvicorn on :9000) — deployment automation +- Exposed Postgres ports 5436/5491/5493 accessible from GCP internal network + +## Related + +- [[wiki/infrastructure/server-optical|server-optical]] — main production server +- [[wiki/infrastructure/ssh-aliases|ssh-aliases]] +- [[wiki/architecture/docker-compose-deploy|docker-compose-deploy]] diff --git a/wiki/infrastructure/server-optical-prod.md b/wiki/infrastructure/server-optical-prod.md new file mode 100644 index 0000000..f100c93 --- /dev/null +++ b/wiki/infrastructure/server-optical-prod.md @@ -0,0 +1,37 @@ +--- +tags: [infrastructure, server, gcp, oliver-internal] +updated: 2026-04-24 +--- + +# optical-prod — Secondary Production (Minimal) + +> SSH alias: `optical-prod` → `vadym.samoilenko@10.220.168.8:22` +> Key: `~/.ssh/id_rsa_vadym` + +## Overview + +Appears to be a GCP instance that is either newly provisioned or used for a very specific purpose. Currently running very few services — only ai_qc and the base OS stack. + +- **Platform**: GCP +- **OS**: Ubuntu (likely 24.04) +- **IP**: 10.220.168.8 +- **Web server**: Apache on 80/443 (configuration unknown — no sites-enabled found) +- **Disk**: 532 GB / 288 GB used (55%) + +## Running + +- Apache on :80 / :443 +- ssh on :22 +- `/opt/`: `ai_qc`, `containerd`, `google-cloud-ops-agent` + +## Key Takeaways + +- Minimal server — possibly being set up or reserved for scaling +- ai_qc is present in `/opt/` but no Docker containers running +- No services beyond OS baseline +- **⚠ Clarify purpose before deploying anything here** + +## Related + +- [[wiki/infrastructure/server-optical|server-optical]] — main prod +- [[wiki/infrastructure/server-optical-dev|server-optical-dev]] — dev/staging diff --git a/wiki/infrastructure/server-optical.md b/wiki/infrastructure/server-optical.md new file mode 100644 index 0000000..b5fda04 --- /dev/null +++ b/wiki/infrastructure/server-optical.md @@ -0,0 +1,115 @@ +--- +tags: [infrastructure, server, gcp, oliver-internal] +updated: 2026-04-24 +--- + +# optical — optical-web-1 (Main AI Production) + +> SSH alias: `optical` → `vadym.samoilenko@10.220.168.5:22` +> Key: `~/.ssh/id_rsa_vadym` + +## Overview + +Main Oliver Agency AI production server. Hosts ~35 client and internal AI applications as systemd services (mix of Python/FastAPI, Node.js, Docker Compose). The biggest server in the fleet — many apps share resources on a single GCP instance. + +- **Hostname**: optical-web-1 +- **Platform**: GCP (europe-west2 assumed) +- **OS**: Ubuntu 22.04.4 LTS +- **IP**: 10.220.168.5 +- **Web server**: Apache 2.4 (no Nginx) +- **Domain**: ai-sandbox.oliver.solutions +- **Disk**: 116 GB total + +## Running Applications (`/opt/`) + +| App | Service | Port | Tech | +|-----|---------|------|------| +| agent_collector | agent_collector.service | — | Python | +| AgentHub / agent_tracker | agent_tracker.service | 17000 | Python/FastAPI | +| AI QC | ai_qc.service | — | Python | +| Box→Bynder | box2bynder.service | — | Python | +| Brief Extractor | brief-extractor.service | — | Python | +| Cannes Video Generator | cannes-video-generator.service | — | Python | +| Contract Query | contract-query.service | — | Python | +| Ferrero CreativeX | creativex-service.service | — | Python | +| Ferrero AC Booking | ferrero-ac.service | — | Python | +| Ferrero Orchestrator | ferrero-orchestrator-prod.service | — | Python | +| Hiring Calculator | hiring_calculator.service | 8127 | Flask/Gunicorn | +| H&M EMS | hm-ems.service | 3001 (Node) | Node.js | +| HP Chatbot | hp_chatbot.service | — | Python | +| JustEight | justeight.service | 7395 | Python | +| Markdown→PDF | markdown2pdf.service | — | Python | +| MSFT Melanie Bot | melanie_bot.service | 8475/8505 | Python | +| Netflix Chatbot | netflix_chatbot.service | — | Python | +| Netflix V2 | netflix_v2.service | — | Python | +| PDF Extract | pdf_extract.service | 8746 | Python | +| Semblance | semblance.service | 8001 | Python | +| Veo Video Generator | veo-video-generator.service | 7394 | Python | +| Video Optimizer | video-optimizer-backend.service | 8003 | Python | +| Video Query | video_query.service | 8000 | Python | +| Voice→Text (Whisper) | voice2text-api.service | 8100 | Python | +| PM2 apps (vadym) | pm2-vadym.samoilenko.service | 3000, 4000, 3100, 3456 | Node.js | + +**Other `/opt/` dirs**: apac-ops-bot, apac-strategy-dashboard, build-a-squad, enterprise-ai-hub-nexus, ferrero-opentext, hm-ems-data, hm_ai_qc, hm_ems_report, loreal-sla-calculator, lusa-back-planner, mongodb, oliver-metadata-tool, ollie-brandtech-strategic-intelligence, pimco-charts, rackham-meeting-analyzer, sandbox-notebookllamalm-nextjs, smartcrop26, veo3, video-accessibility, video-subtitle, backups, agent-sync + +## Port Map + +| Port | Service | Notes | +|------|---------|-------| +| 80 / 443 | Apache | ai-sandbox.oliver.solutions | +| 3000 | PM2 / Node.js | — | +| 3001 | H&M EMS Node | — | +| 3100 | PM2 / Node.js | — | +| 3456 | Node.js (fac) | ProxyPass /fac/ → :3456 | +| 4000 | PM2 / Node.js | — | +| 5000 | Gunicorn (localhost) | — | +| 5432 / 5433 / 5437 | PostgreSQL | Three instances | +| 6174 / 6175 / 6176 | Redis internal | — | +| 6333 | Qdrant | Vector DB (internal: 10.220.168.5:6333) | +| 6379 / 6380 / 6399 | Redis | Three instances | +| 7474 / 7475 | Neo4j HTTP | — | +| 7687 / 7688 | Neo4j Bolt | — | +| 7394 / 7395 | JustEight | — | +| 8000 | Video Query | — | +| 8001 | Semblance | — | +| 8003 | Video Optimizer | — | +| 8038 / 8048 | Various | — | +| 8080 | — | — | +| 8100 | Voice2Text | — | +| 8127 | Hiring Calculator | — | +| 8475 / 8505 | Melanie Bot | — | +| 8746 | PDF Extract | — | +| 8877 / 8569 | Internal only (localhost) | — | +| 9000 / 9001 | — | — | +| 17000 | AgentHub | — | +| 27017 / 27019 / 27021 | MongoDB | Three instances | + +## Databases Running + +- **PostgreSQL**: :5432, :5433, :5437 (three instances) +- **MongoDB**: :27017, :27019, :27021 (three instances) +- **Redis**: :6379, :6380, :6399 + internal on 6174–6176 +- **Qdrant** (vector DB): :6333 (bound to 10.220.168.5 only) +- **Neo4j**: :7474/:7475 (HTTP), :7687/:7688 (Bolt) +- Also: internal Redis exposed to optical-dev via :15432 (PG) and :16379 (Redis) + +## Apache Config + +- Single vhost: `ai-sandbox.oliver.solutions` +- SSL termination: upstream (GCP load balancer) +- PHP 8.1-FPM running alongside +- ProxyPass: `/fac/` → `http://127.0.0.1:3456/` + +## Key Takeaways + +- This is the **main workhorse** — don't restart services carelessly +- Multiple databases of each type — always confirm which port before connecting +- Apps deployed in `/opt/{name}/` as systemd services — manage with `sudo systemctl {start|stop|status} {service-name}` +- Also some Docker Compose apps (loreal-sla-calculator, enterprise-ai-hub-nexus, video-accessibility, pdf-accessibility, pimco-charts, ac-helper, sandbox apps) +- PM2 manages Node.js apps under `vadym.samoilenko` user + +## Related + +- [[wiki/infrastructure/server-optical-dev|server-optical-dev]] — Docker staging server +- [[wiki/infrastructure/ssh-aliases|ssh-aliases]] — all server quick reference +- [[wiki/architecture/docker-compose-deploy|docker-compose-deploy]] diff --git a/wiki/infrastructure/server-pve.md b/wiki/infrastructure/server-pve.md new file mode 100644 index 0000000..188b19a --- /dev/null +++ b/wiki/infrastructure/server-pve.md @@ -0,0 +1,133 @@ +--- +tags: [infrastructure, server, proxmox, homelab, personal] +updated: 2026-04-24 +--- + +# pve — Proxmox VE Homelab + +> SSH alias: `pve` → `root@192.168.1.48:22` +> Key: `~/.ssh/id_ed25519` +> Web UI: https://192.168.1.48:8006 + +## Overview + +Home Proxmox VE server. Runs VMs and LXC containers for personal projects, self-hosted services, and homelab experimentation. Connected via Tailscale for remote access. + +- **Platform**: Bare-metal (home server) +- **OS**: Proxmox VE 9.1.7 (kernel 6.17.13-2-pve) +- **IP**: 192.168.1.48 (LAN) +- **Tailscale**: 100.122.192.8 (accessible remotely) +- **CPU/RAM**: Not audited (runs 8 containers + 1 VM comfortably) + +## Storage + +| Pool | Type | Total | Used | Available | % | +|------|------|-------|------|-----------|---| +| data-hdd | LVM-thin | 5.6 TB | 25 GB | 5.5 TB | 0.4% | +| local | dir | 68 GB | 6.5 GB | 58 GB | 9% | +| local-lvm | LVM-thin | 141 GB | 100 GB | 41 GB | 71% | +| usb-backup | dir | 916 GB | 111 GB | 759 GB | 12% | + +⚠ **local-lvm is 71% full** — watch this pool + +## Virtual Machines + +| VMID | Name | Status | RAM | Disk | +|------|------|--------|-----|------| +| 200 | kali-linux | running | 8 GB | 60 GB | + +## LXC Containers + +| VMID | Name | Status | Purpose | +|------|------|--------|---------| +| 100 | ollama | running | Local LLM inference (Ollama) | +| 101 | adguard | running | DNS ad-blocking (AdGuard Home) | +| 102 | docker | running | General Docker container host | +| 103 | beszel | running | Server monitoring dashboard | +| 104 | vaultwarden | running | Bitwarden-compatible password manager | +| 105 | immich | running | Self-hosted photo management | +| 107 | homarr | running | Homepage / service dashboard | +| 110 | uptimekuma | running | Uptime monitoring | + +## Host Ports + +| Port | Service | +|------|---------| +| 8006 | Proxmox Web UI (HTTPS) | +| 3128 | SPICE proxy | +| 22 | SSH | +| 45876 | Beszel agent | +| 9101 | node_exporter (Prometheus metrics) | + +## Key Services on Host + +- **Tailscale** — remote access overlay (100.122.192.8) +- **Beszel agent** — system monitoring +- **node_exporter** — Prometheus metrics +- **Postfix** — local mail relay + +## Container Details + +### CT 100 — ollama +- Runs Ollama server for local LLM inference +- Access models from LAN + +### CT 101 — adguard +- AdGuard Home DNS server +- Likely set as LAN DNS resolver (192.168.1.x) + +### CT 102 — docker +- General-purpose Docker host for misc containers +- Check inside with `pct exec 102 -- docker ps` + +### CT 103 — beszel +- Monitoring hub for all containers/VMs +- Pulls metrics from beszel agents running on each host + +### CT 104 — vaultwarden +- Self-hosted Bitwarden password manager +- Separate from aimpress VPS vaultwarden + +### CT 105 — immich +- Photo management (Google Photos alternative) + +### CT 107 — homarr +- Dashboard at http://192.168.1.224:7575 (or configured port) +- Shows all services, integrates with Docker/Proxmox + +### CT 110 — uptimekuma +- Monitors all services for uptime +- Web UI — configure webhooks to monitoring-agent for alerts +- ⚠ Pending: add Webhook to `http://192.168.1.225:9111/alert/uptime-kuma` + +## Useful Commands + +```bash +# List VMs and containers +ssh pve "qm list && pct list" + +# Execute command in container +ssh pve "pct exec 102 -- docker ps" + +# Start/stop container +ssh pve "pct start 110" +ssh pve "pct stop 110" + +# Check storage +ssh pve "pvesm status" +``` + +## Key Takeaways + +- **local-lvm at 71%** — clean up unused volumes or expand before hitting 85% +- Kali Linux VM (200) running — for security/pentesting +- Ollama (CT 100) = local LLM inference, no API key needed +- AdGuard (CT 101) = DNS for LAN — changing it affects all home devices +- All containers running — healthy cluster +- Tailscale enables access from anywhere without port forwarding + +## Related + +- [[wiki/homelab/_index|homelab/]] — full homelab docs +- [[wiki/infrastructure/server-aimpress|server-aimpress]] +- [[wiki/infrastructure/ssh-aliases|ssh-aliases]] diff --git a/wiki/infrastructure/ssh-aliases.md b/wiki/infrastructure/ssh-aliases.md new file mode 100644 index 0000000..4383394 --- /dev/null +++ b/wiki/infrastructure/ssh-aliases.md @@ -0,0 +1,64 @@ +--- +tags: [infrastructure, ssh, reference] +updated: 2026-04-24 +--- + +# SSH Aliases — Quick Reference + +All servers accessible from `~/.ssh/config`. All confirmed reachable as of 2026-04-24. + +## Oliver Agency — GCP Servers + +| Alias | Hostname/IP | User | Purpose | +|-------|------------|------|---------| +| `optical` | 10.220.168.5 | vadym.samoilenko | Main AI prod server (optical-web-1, ai-sandbox.oliver.solutions) | +| `optical-dev` | 10.220.168.9 | vadym.samoilenko | Docker staging (optical-dev.oliver.solutions) | +| `optical-prod` | 10.220.168.8 | vadym.samoilenko | Secondary prod (minimal, ai_qc only) | +| `librechat-dev` | 10.220.168.2 | vadym.samoilenko | LibreChat dev (optical-librechat-dev) | +| `librechat-prod` | 10.220.168.4 | vadym.samoilenko | LibreChat production | +| `modocmms-dev` | 10.220.168.6 | vadym.samoilenko | ModoCMMS dev+prod (modcomms-01) | +| `baic` | 10.220.72.13 | vadym.samoilenko | Main web host — 40+ domains (web-03) | +| `box-cli` | 10.220.176.3 | vadym.samoilenko | Ford/L'Oréal hotfolder processing (box-cli-01, CentOS 7) | + +## Personal / Aimpress + +| Alias | Hostname/IP | Port | User | Purpose | +|-------|------------|------|------|---------| +| `aimpress` | 57.128.160.249 | 1220 | ubuntu | Aimpress LTD VPS (Mailcow, n8n, Traefik) | +| `pve` | 192.168.1.48 | 22 | root | Proxmox homelab (LAN only + Tailscale) | + +## SSH Keys Used + +| Key | Used For | +|-----|---------| +| `~/.ssh/id_rsa_vadym` | All GCP Oliver servers | +| `~/.ssh/id_rsa` | aimpress VPS | +| `~/.ssh/id_ed25519` | Proxmox homelab | +| `~/.ssh/id_bitbucket` | Bitbucket git (git@bitbucket.org) | + +## One-liner Health Check + +```bash +for host in optical optical-dev optical-prod librechat-dev librechat-prod modocmms-dev baic box-cli aimpress pve; do + ssh -o ConnectTimeout=5 -o BatchMode=yes "$host" "echo OK" 2>&1 | grep -q OK && echo "✅ $host" || echo "❌ $host" +done +``` + +## Key Takeaways + +- All 10 hosts accessible with SSH keys (no password prompts) +- `baic` alias is misleading — it's the main Oliver web server, not just BAIC client +- `box-cli` is CentOS 7 (EOL) — plan migration +- `pve` is LAN only (192.168.1.48) — use Tailscale (100.122.192.8) when remote +- `aimpress` uses non-standard port 1220 + +## Related + +- [[wiki/infrastructure/server-optical|server-optical]] +- [[wiki/infrastructure/server-optical-dev|server-optical-dev]] +- [[wiki/infrastructure/server-baic|server-baic]] +- [[wiki/infrastructure/server-box-cli|server-box-cli]] +- [[wiki/infrastructure/server-aimpress|server-aimpress]] +- [[wiki/infrastructure/server-pve|server-pve]] +- [[wiki/infrastructure/server-librechat|server-librechat]] +- [[wiki/infrastructure/server-modocmms|server-modocmms]] diff --git a/wiki/log.md b/wiki/log.md index f32ad91..ce457c5 100644 --- a/wiki/log.md +++ b/wiki/log.md @@ -38,6 +38,28 @@ - Articles created: [[wiki/concepts/export-endpoint-filter-pattern]] - Articles updated: [[wiki/concepts/_index]] (concepts count 23→24), [[wiki/_master-index]] (concepts 23→24) +## [2026-04-24T11:30:00+01:00] compile | 2026-04-21.md +- Source: daily/2026-04-21.md +- Articles created: [[wiki/concepts/ollama-lxc-ram-requirements]], [[wiki/concepts/beszel-monitoring-deployment]], [[wiki/concepts/adguard-dns-rewrites-homelab]], [[wiki/concepts/prometheus-joules-watts-gotcha]], [[wiki/concepts/old-gpu-sysfs-metrics]], [[wiki/concepts/chartjs-time-axis-adapter]], [[wiki/concepts/uptime-kuma-socketio-management]], [[wiki/concepts/proxmox-container-502-misdiagnosis]] +- Articles updated: [[wiki/concepts/_index]] (concepts 24→32), [[wiki/_master-index]] (concepts 24→32) + +## [2026-04-24T11:45:00+01:00] compile | 2026-04-22.md +- Source: daily/2026-04-22.md +- Articles created: (none) +- Articles updated: (none) +- Note: Log contained only flush errors (exit code 1) — no session content to compile + +## [2026-04-24T12:00:00+01:00] compile | 2026-04-23.md +- Source: daily/2026-04-23.md +- Articles created: [[wiki/concepts/git-includeif-per-remote]], [[wiki/concepts/dns-youtube-ad-blocking]], [[wiki/concepts/adguard-blocklist-setup]] +- Articles updated: [[wiki/concepts/_index]] (concepts 32→35), [[wiki/_master-index]] (concepts 32→35) +- Note: Grafana LXC CT109 permanently deleted — noted in session but no article changes needed (beszel article still reflects correct architecture) + +## [2026-04-24T13:00:00+01:00] compile | 2026-04-24.md +- Source: daily/2026-04-24.md +- Articles created: [[wiki/concepts/claude-code-schedule-skill-account-type]], [[wiki/concepts/memory-compiler-mac-migration]] +- Articles updated: [[wiki/concepts/_index]] (concepts 35→37), [[wiki/_master-index]] (concepts 35→37) + ## [2026-04-17T16:03:42+01:00] compile | 2026-04-15.md - Source: daily/2026-04-15.md - Articles created: [[wiki/concepts/librechat-openid-auth]], [[wiki/concepts/msal-vanilla-js-pkce]], [[wiki/concepts/shell-static-deploy-patterns]], [[wiki/concepts/fish-shell-path-config]], [[wiki/connections/oauth-state-mismatch-debugging]]