# 🏗️ Complete Infrastructure Report for AI-Impress

> **Generated:** $(date +'%Y-%m-%d %H:%M:%S')
> **Host:** $(hostname) | **IP:** 51.89.231.46
> **Status:** $(docker ps -q | wc -l) containers running

---

## 📑 TABLE OF CONTENTS

1. [Executive Summary](#executive-summary) - Quick overview
2. [Service Inventory](#service-inventory) - What we have and why
3. [Architecture](#architecture) - How it all connects
4. [Storage & Locations](#storage--locations) - Where everything is
5. [Access & Credentials](#access--credentials) - Logins and passwords
6. [How to Manage](#how-to-manage) - Step-by-step instructions
7. [Current Status](#current-status) - What's working now
8. [Troubleshooting](#troubleshooting) - Common problems and fixes

---

## 1️⃣ EXECUTIVE SUMMARY

### Quick Health Check
- 🐳 **Docker Containers:** $(docker ps -q | wc -l) running / $(docker ps -aq | wc -l) total
- 💾 **Storage:** $(du -sh /mnt/backups 2>/dev/null | cut -f1) in backups, $(du -sh /mnt/psql-data 2>/dev/null | cut -f1) in databases
- 🌐 **Networks:** $(docker network ls -q | wc -l) Docker networks
- 📦 **Volumes:** $(docker volume ls -q | wc -l) volumes

### What This System Does
This is a complete business automation platform with:
- **Wiki & Documentation** (Outline)
- **Workflow Automation** (N8N)
- **CRM & Marketing** (Mautic)
- **Business Management** (Odoo)
- **Team Chat & Auth** (Authentik)
- **File Storage** (Supabase)
- **Password Manager** (Vaultwarden)
- **Backups & Recovery** (Restic to Cloudflare R2)

---

## 2️⃣ SERVICE INVENTORY

### 🏛️ Infrastructure Core (CRITICAL - Cannot Delete!)
These services MUST stay running or everything breaks:

| Service | Container | Purpose | Storage | Port |
|---------|-----------|---------|---------|------|
| **Traefik** | traefik | Reverse proxy, SSL, routing | None | 80, 443, 8080 |
| **PostgreSQL** | postgres-main | Main database (N8N, Outline, Odoo, etc) | /mnt/psql-data | 5432 |
| **Redis** | redis-main | Cache & queues | Docker volume | 6379 |
| **Vault** | vault | Secrets management | /opt/00-infrastructure/vault | 8200 |

**IF ANY OF THESE GO DOWN - EVERYTHING STOPS!**

### 🌐 Network Services (IMPORTANT - Handle with Care)

| Service | Container | Purpose | Users | URL |
|---------|-----------|---------|-------|-----|
| **Authentik** | authentik-* | Single sign-on, user management | N/A (SSO) | https://auth.ai-impress.com |
| **RabbitMQ** | rabbitmq | Message broker | guest / password | https://rabbitmq.ai-impress.com |
| **PgAdmin** | pgadmin | Database management UI | admin@admin.com | https://pgadmin.ai-impress.com |
| **Portainer** | portainer | Docker management UI | admin | https://portainer.ai-impress.com |

### 💼 Business Applications (Can be removed individually)

#### Documentation & Knowledge
| Service | Container | Purpose | Users | URL | Storage |
|---------|-----------|---------|-------|-----|---------|
| **Outline** | outline | Wiki, documentation | Use Authentik SSO | https://wiki.ai-impress.com | PostgreSQL |
| **Wiki.js** | wikijs | Alternative wiki | (disabled) | https://info.ai-impress.com | PostgreSQL |
| **Documenso** | documenso | Document signing | signup | https://sign.ai-impress.com | PostgreSQL |

#### Automation & Workflows
| Service | Container | Purpose | Users | URL | Storage |
|---------|-----------|---------|-------|-----|---------|
| **N8N** | n8n-shared, n8n-worker-* | Workflow automation | Use Authentik SSO | https://n8n.ai-impress.com | PostgreSQL + Volumes |
| **Mautic** | mautic, mautic-db | Marketing automation | admin | https://marketing.ai-impress.com | MariaDB |
| **Evolution API** | evolution-api | WhatsApp integration | API key based | https://wpp.ai-impress.com | PostgreSQL |

#### Business Management
| Service | Container | Purpose | Users | URL | Storage |
|---------|-----------|---------|-------|-----|---------|
| **Odoo** | odoo | ERP system | admin | https://odoo.ai-impress.com | PostgreSQL |
| **Postiz** | postiz-* | Social media manager | Login required | https://social.ai-impress.com | PostgreSQL |

#### Security & Access
| Service | Container | Purpose | Users | URL | Storage |
|---------|-----------|---------|-------|-----|---------|
| **Vaultwarden** | vaultwarden | Password manager | Personal accounts | https://vault.ai-impress.com | PostgreSQL |
| **Supabase** | supabase-* | Backend as a service | API key | https://supabase.ai-impress.com | PostgreSQL |

#### Monitoring
| Service | Container | Purpose | Users | URL | Storage |
|---------|-----------|---------|-------|-----|---------|
| **Grafana** | grafana | Metrics dashboards | admin | https://grafana.ai-impress.com | Volumes |
| **Prometheus** | prometheus | Metrics collection | N/A | Internal only | Volumes |
| **Uptime Kuma** | uptime-kuma | Uptime monitoring | admin | https://status.ai-impress.com | Volumes |

---

## 3️⃣ ARCHITECTURE

### 🔗 How Services Connect

```
INTERNET (users)
    ↓
443/80 (HTTPS)
    ↓
┌──────────────────┐
│  TRAEFIK         │ ← Reverse Proxy, SSL, Routing
│  (Routing Rules) │
└──────────────────┘
    ↓
    ├─→ auth.ai-impress.com → authentik-server
    ├─→ wiki.ai-impress.com → outline
    ├─→ n8n.ai-impress.com → n8n-shared
    ├─→ marketing.ai-impress.com → mautic
    ├─→ odoo.ai-impress.com → odoo
    └─→ ...other services...
    ↓
┌──────────────────────────────┐
│  DOCKER NETWORKS             │
├──────────────────────────────┤
│ traefik-public               │ ← All web services
│ database-internal            │ ← DB access
│ authentik_authentik-internal │ ← Authentik only
│ n8n-shared                   │ ← N8N cluster
│ supabase_default             │ ← Supabase stack
└──────────────────────────────┘
    ↓
┌─────────────────────────────────────────┐
│  CORE INFRASTRUCTURE                     │
├─────────────────────────────────────────┤
│ PostgreSQL Main (postgres-main)          │
│ ├─ n8n_shared                           │
│ ├─ outline                              │
│ ├─ odoo                                 │
│ ├─ documenso                            │
│ ├─ vaultwarden                          │
│ └─ ...others...                         │
│                                          │
│ MariaDB (mautic-db)                     │
│ ├─ Mautic application                   │
│ └─ Integration data                     │
│                                          │
│ Redis Main (redis-main)                 │
│ ├─ N8N queue management                 │
│ ├─ Outline sessions                     │
│ └─ Cache                                │
│                                          │
│ Vault                                   │
│ ├─ All passwords & tokens               │
│ └─ Encryption keys                      │
└─────────────────────────────────────────┘
```

### 🌐 Docker Networks Explained

**traefik-public** (Public services)
- Containers exposed via Traefik
- Can reach HTTPS endpoints
- Examples: outline, n8n, mautic, odoo

**database-internal** (Database access)
- Services that need database
- Cannot reach public internet directly
- Examples: n8n, outline, odoo, vaultwarden

**authentik_authentik-internal** (SSO only)
- Only Authentik services
- Isolated for security

**n8n-shared** (Workflow cluster)
- Main N8N + worker nodes
- Share queue and database

---

## 4️⃣ STORAGE & LOCATIONS

### 📁 Folder Structure

```
/opt/                              # ALL APPLICATION DATA
├── 00-infrastructure/             # Core infra
│   ├── vault/                     # Vault secrets & config
│   ├── traefik/                   # Traefik config
│   └── docker-compose.yml
├── 01-network/                    # Network services
│   ├── authentik/
│   ├── rabbitmq/
│   └── docker-compose.yml
├── 02-core/                       # Core services
│   ├── postgres/                  # PostgreSQL
│   ├── redis/                     # Redis cache
│   ├── n8n-shared/                # N8N automation
│   ├── outline/                   # Wiki
│   ├── supabase/                  # Backend as service
│   └── docker-compose.yml
├── 03-business/                   # Business apps
│   ├── mautic/                    # Marketing automation
│   ├── odoo/                      # ERP
│   ├── vaultwarden/               # Password manager
│   └── docker-compose.yml
├── 04-ai/                         # AI services
│   ├── open-webui/
│   └── docker-compose.yml
├── 05-backups/                    # BACKUPS & SCRIPTS
│   ├── scripts/                   # Management scripts
│   ├── reports/                   # Generated reports
│   ├── logs/                      # Script logs
│   ├── data/                      # Reference data
│   └── backups/                   # Old app backups
└── infrastructure-docs/           # Documentation

/mnt/                              # EXTERNAL STORAGE
├── backups/                       # ALL BACKUPS (Restic + local)
│   ├── postgresql-*.sql.gz        # DB backups
│   ├── mariadb-*.sql.gz           # Mautic DB backups
│   ├── vault-data-*.tar.gz        # Vault backups
│   ├── docker-configs-*.tar.gz    # Config files
│   └── app-data-*.tar.gz          # App data
└── psql-data/                     # DATABASE STORAGE
    ├── pgdata/                    # PostgreSQL data files
    └── mautic-db/                 # MariaDB data files
```

### 💾 Volumes & Storage Mapping

| Volume Name | Location | Size | Used By | Backup |
|-------------|----------|------|---------|--------|
| postgres_data | /var/lib/docker/volumes/.../data | 50GB | PostgreSQL | ✅ Daily |
| redis_data | /var/lib/docker/volumes/.../data | 2GB | Redis | ❌ No backup |
| n8n-data | /var/lib/docker/volumes/.../data | 1GB | N8N | ✅ Restic |
| mautic_db | /mnt/psql-data/mautic-db | 30GB | MariaDB | ✅ Daily |
| outline_data | /var/lib/docker/volumes/.../data | 500MB | Outline | ✅ Restic |

**BACKUP STRATEGY:**
- PostgreSQL: Daily SQL dump + Restic
- MariaDB: Daily SQL dump + Restic
- Vault: Daily tar backup + Restic
- Configs: Version controlled + Restic
- Restic: Upload to Cloudflare R2 (offsite)

---

## 5️⃣ ACCESS & CREDENTIALS

### 🔐 Where Passwords Are Stored

**Primary:** Vault at `/opt/00-infrastructure/vault/`
**Secondary:** Bitwarden (synced from Vault)

### 🌐 Web Services Access

| Service | URL | Username | Password Location | Notes |
|---------|-----|----------|-------------------|-------|
| Outline | https://wiki.ai-impress.com | Use Authentik SSO | N/A | Create account via SSO |
| N8N | https://n8n.ai-impress.com | Use Authentik SSO | N/A | Create account via SSO |
| Mautic | https://marketing.ai-impress.com | admin | `vault kv get aimpress/mautic` | Separate DB |
| Odoo | https://odoo.ai-impress.com | admin | `vault kv get aimpress/odoo` | ERP system |
| Authentik | https://auth.ai-impress.com | N/A (SSO) | N/A | Admin: `vault kv get aimpress/authentik` |
| Vaultwarden | https://vault.ai-impress.com | Personal | Personal account | Password manager |
| Portainer | https://portainer.ai-impress.com | admin | `vault kv get aimpress/portainer` | Docker management |
| Grafana | https://grafana.ai-impress.com | admin | `vault kv get aimpress/grafana` | Monitoring |
| PgAdmin | https://pgadmin.ai-impress.com | admin@admin.com | `vault kv get aimpress/pgadmin` | Database UI |

### 🗄️ Database Access

```bash
# PostgreSQL
HOST: postgres-main (internal) or localhost:5432 (from host)
PORT: 5432
ADMIN USER: aimpress_admin
ADMIN PASSWORD: vault kv get aimpress/postgres/admin

# Get specific app password:
vault kv get aimpress/postgres/outline
vault kv get aimpress/postgres/n8n
vault kv get aimpress/postgres/odoo

# Connect example:
docker exec -it postgres-main psql -U aimpress_admin

# MariaDB (Mautic)
HOST: mautic-db (internal)
PORT: 3306
ADMIN USER: root
ADMIN PASSWORD: vault kv get aimpress/mautic/db_password

# Redis
HOST: redis-main (internal)
PORT: 6379
PASSWORD: vault kv get aimpress/redis/password
```

---

## 6️⃣ HOW TO MANAGE

### ✏️ Change Password Example

#### For PostgreSQL database user (outline):

```bash
# Step 1: SSH into server
ssh ubuntu@51.89.231.46

# Step 2: Generate new password
NEW_PASS=$(openssl rand -base64 32)
echo "New password: $NEW_PASS"

# Step 3: Change in database
docker exec postgres-main psql -U aimpress_admin -c "ALTER USER outline_user WITH PASSWORD '$NEW_PASS';"

# Step 4: Update in Vault
vault kv put aimpress/postgres/outline password=$NEW_PASS

# Step 5: Restart application
cd /opt/02-core/outline && docker-compose restart outline

# Step 6: Verify
curl -s https://wiki.ai-impress.com | head -20
```

### 🗑️ Delete Unused Service Example

#### Remove a container you don't need:

```bash
# Step 1: Check what will be affected
docker ps | grep documenso

# Step 2: Backup data first!
docker exec documenso mysqldump --all-databases > /tmp/documenso-backup.sql

# Step 3: Stop the service
docker-compose -f /opt/03-business/docker-compose.yml stop documenso

# Step 4: Remove container
docker rm documenso

# Step 5: Remove volumes (if desired)
docker volume rm documenso_data

# Step 6: Clean docker-compose file
nano /opt/03-business/docker-compose.yml  # Remove documenso section
docker-compose -f /opt/03-business/docker-compose.yml up -d

# Step 7: Verify removal
docker ps | grep documenso  # Should show nothing
```

### 🔄 Restart Services

```bash
# Restart single service
docker restart <container_name>

# Restart all services in project
cd /opt/02-core && docker-compose restart

# Restart critical service (PostgreSQL)
cd /opt/02-core/postgres && docker-compose restart postgres-main

# Verify health after restart
docker ps | grep postgres-main
```

### 📊 Check Service Logs

```bash
# Last 50 lines
docker logs --tail 50 <container_name>

# Follow logs (live)
docker logs -f <container_name>

# Last 100 lines with timestamps
docker logs --tail 100 --timestamps <container_name>

# Example: Check Outline
docker logs -f outline
```

---

## 7️⃣ CURRENT STATUS

### 🟢 Healthy Services
$(docker ps --format "table {{.Names}}\t{{.Status}}" | grep -E "(healthy|Up)" | awk '{print "- **" $1 ":** " substr($0, index($0,$2))}')

### 🔴 Problem Services
$(docker ps --filter health=unhealthy --format "table {{.Names}}\t{{.Status}}" | awk 'NR>1 {print "- **" $1 ":** " $2}')

### 📊 Resource Usage

**CPU:** $(docker stats --no-stream --format "table {{.CPUPerc}}" | tail -1)
**Memory:** $(free -h | grep Mem | awk '{print $3 "/" $2}')
**Disk:** $(df -h / | tail -1 | awk '{print $3 "/" $2 " (" $5 ")"}')

### 🌐 Website Status

$(for domain in wiki.ai-impress.com n8n.ai-impress.com marketing.ai-impress.com odoo.ai-impress.com; do
  if timeout 3 curl -skI "https://$domain" 2>/dev/null | grep -q "HTTP"; then
    echo "- ✅ $domain - Online"
  else
    echo "- ❌ $domain - Offline"
  fi
done)

---

## 8️⃣ TROUBLESHOOTING

### "Website not responding"

```bash
# 1. Check Traefik
docker ps | grep traefik  # Should be "Up"

# 2. Check target container
docker ps | grep <service_name>  # Should be "Up" or "healthy"

# 3. Check DNS
nslookup wiki.ai-impress.com

# 4. Check port
curl -v https://wiki.ai-impress.com

# 5. Check Traefik logs
docker logs traefik | tail -50

# 6. Restart Traefik
docker-compose -f /opt/00-infrastructure/docker-compose.yml restart traefik
```

### "Database connection failed"

```bash
# 1. Check PostgreSQL
docker ps | grep postgres-main

# 2. Check if can connect
docker exec postgres-main psql -U aimpress_admin -c "SELECT 1;"

# 3. Check disk space
df -h /mnt/psql-data  # Must have free space

# 4. Check logs
docker logs postgres-main | tail -50

# 5. Restart if needed
docker-compose -f /opt/02-core/postgres/docker-compose.yml restart postgres-main

# 6. Check connections
docker exec postgres-main psql -U aimpress_admin -c "SELECT datname, count(*) FROM pg_stat_activity GROUP BY datname;"
```

### "Out of disk space"

```bash
# 1. Find large files
du -sh /opt/* /mnt/*

# 2. Check docker
docker system df

# 3. Clean old logs
find /opt/05-backups/logs -mtime +30 -delete

# 4. Clean docker
docker system prune -a  # WARNING: removes unused images!

# 5. Check /mnt/backups for old backups
ls -lh /mnt/backups/ | head -20
# Delete old: rm /mnt/backups/postgresql-*.sql.gz (keep last 10)
```

---

## 📝 Quick Reference

### Most Used Commands

```bash
# Check everything is running
ssh ubuntu@51.89.231.46 "docker ps"

# See report like this one
ssh ubuntu@51.89.231.46 "/opt/infrastructure-docs/format-snapshot-full.sh"

# Manage all with admin tool (when ready)
ssh ubuntu@51.89.231.46 "/opt/05-backups/scripts/admin.sh help"

# Check health
ssh ubuntu@51.89.231.46 "/opt/05-backups/scripts/system-diagnostics.sh"

# Full backup
ssh ubuntu@51.89.231.46 "/opt/05-backups/scripts/backup-full-enhanced.sh"
```

---

## 📞 Support Notes

- **Backups:** Automated daily to /mnt/backups and Cloudflare R2
- **Monitoring:** Grafana dashboard at https://grafana.ai-impress.com
- **Logs:** Centralized in /opt/05-backups/logs/
- **Documentation:** Full docs in Outline Wiki

**Generated:** $(date)
**Host:** $(hostname) (51.89.231.46)

