ideas-generator/INSTALLATION.md

# Ideas Generator 2025 - Installation Guide

A comprehensive installation guide for deploying the Ideas Generator 2025 application with dual-agent system, file upload support, and complete admin dashboard.

## 📋 Table of Contents

1. [Prerequisites](#prerequisites)
2. [Environment Setup](#environment-setup)
3. [Database Configuration](#database-configuration)
4. [Backend Installation](#backend-installation)
5. [Frontend Installation](#frontend-installation)
6. [Environment Variables](#environment-variables)
7. [Database Migration](#database-migration)
8. [Initial Data Setup](#initial-data-setup)
9. [Running the Application](#running-the-application)
10. [Production Deployment](#production-deployment)
11. [Troubleshooting](#troubleshooting)
12. [Maintenance](#maintenance)

## 🔧 Prerequisites

### System Requirements
- **Node.js**: 18.0.0 or higher
- **npm**: 9.0.0 or higher
- **PostgreSQL**: 14.0 or higher
- **Git**: Latest version

### Required Services
- **OpenAI API Account** with access to:
  - Chat Completions API
  - Responses API
  - File Upload API
  - Vector Stores (optional)

### Development Tools (Optional)
- **pgAdmin** or similar PostgreSQL management tool
- **Postman** for API testing
- **VS Code** with recommended extensions

## 🌍 Environment Setup

### 1. Clone the Repository

```bash
git clone https://bitbucket.org/zlalani/ideas-generator.git
cd ideas-generator
git checkout ideas-gen-2025
```

### 2. Verify Node.js Installation

```bash
node --version  # Should be 18.0.0+
npm --version   # Should be 9.0.0+
```

### 3. Install Global Dependencies

```bash
npm install -g nodemon
npm install -g pm2  # For production deployment
```

## 🗄️ Database Configuration

### 1. Install PostgreSQL

**macOS (using Homebrew):**
```bash
brew install postgresql@14
brew services start postgresql@14
```

**Ubuntu/Debian:**
```bash
sudo apt update
sudo apt install postgresql postgresql-contrib
sudo systemctl start postgresql
sudo systemctl enable postgresql
```

**Windows:**
Download and install from [PostgreSQL Official Site](https://www.postgresql.org/download/windows/)

### 2. Create Database and User

```bash
# Connect to PostgreSQL as superuser
sudo -u postgres psql

# Create database and user
CREATE DATABASE ideas_gen_dev;
CREATE USER ideas_gen_user WITH ENCRYPTED PASSWORD 'your_secure_password';
GRANT ALL PRIVILEGES ON DATABASE ideas_gen_dev TO ideas_gen_user;

# For production, create separate database
CREATE DATABASE ideas_gen_prod;
GRANT ALL PRIVILEGES ON DATABASE ideas_gen_prod TO ideas_gen_user;

\q
```

### 3. Test Database Connection

```bash
psql -h localhost -U ideas_gen_user -d ideas_gen_dev
```

## 🚀 Backend Installation

### 1. Navigate to Server Directory

```bash
cd server
```

### 2. Install Dependencies

```bash
npm install
```

### 3. Install Additional Dependencies (if needed)

```bash
npm install --save sequelize pg pg-hstore
npm install --save express cors helmet morgan
npm install --save openai multer node-fetch
npm install --save-dev nodemon
```

## 🎨 Frontend Installation

### 1. Navigate to Admin Directory

```bash
cd ../admin
```

### 2. Install Dependencies

```bash
npm install
```

### 3. Install Additional Dependencies (if needed)

```bash
npm install --save vue@next vue-router@4
npm install --save axios chart.js
npm install --save marked
npm install --save-dev @vitejs/plugin-vue vite
```

## 🔐 Environment Variables

### 1. Server Environment (.env)

Create `server/.env` file:

```env
# Application Settings
NODE_ENV=development
PORT=3000
FRONTEND_URL=http://localhost:5173

# Database Configuration
DB_NAME=ideas_gen_dev
DB_USER=ideas_gen_user
DB_PASSWORD=your_secure_password
DB_HOST=localhost
DB_PORT=5432
DB_DIALECT=postgres

# OpenAI API Configuration
OPENAI_API_KEY=sk-your-openai-api-key-here
OPENAI_ORG_ID=org-your-organization-id-here

# Security Settings
JWT_SECRET=your-super-secure-jwt-secret-key-here
SESSION_SECRET=your-super-secure-session-secret-here

# File Upload Settings
MAX_FILE_SIZE=10485760
UPLOAD_PATH=./uploads

# CORS Settings
CORS_ORIGIN=http://localhost:5173

# Logging
LOG_LEVEL=debug
```

### 2. Frontend Environment (.env)

Create `admin/.env` file:

```env
# API Configuration
VITE_API_BASE_URL=http://localhost:3000
VITE_API_TIMEOUT=30000

# Application Settings
VITE_APP_NAME=Ideas Generator 2025
VITE_APP_VERSION=2.0.0

# Development Settings
VITE_DEV_TOOLS=true
VITE_LOG_LEVEL=debug
```

### 3. Production Environment Variables

For production, create `server/.env.production`:

```env
# Application Settings
NODE_ENV=production
PORT=3000
FRONTEND_URL=https://your-domain.com

# Database Configuration
DB_NAME=ideas_gen_prod
DB_USER=ideas_gen_user
DB_PASSWORD=your_production_password
DB_HOST=your-db-host
DB_PORT=5432
DB_DIALECT=postgres
DB_SSL=true

# OpenAI API Configuration
OPENAI_API_KEY=sk-your-production-openai-key
OPENAI_ORG_ID=org-your-production-org-id

# Security Settings (Generate strong secrets!)
JWT_SECRET=your-production-jwt-secret
SESSION_SECRET=your-production-session-secret

# File Upload Settings
MAX_FILE_SIZE=10485760
UPLOAD_PATH=/var/www/uploads

# CORS Settings
CORS_ORIGIN=https://your-domain.com

# Logging
LOG_LEVEL=info
```

## 🗄️ Database Migration

### 1. Run Database Migrations

```bash
cd server
npm run migrate
```

### 2. If migrations don't exist, sync models

```bash
cd server
node -e "
const { sequelize } = require('./models');
sequelize.sync({ force: false })
  .then(() => console.log('✅ Database synced'))
  .catch(err => console.error('❌ Sync failed:', err));
"
```

### 3. Verify Tables Created

```bash
psql -h localhost -U ideas_gen_user -d ideas_gen_dev -c "\dt"
```

Expected tables:
- users
- assistants
- conversations
- messages
- response_sessions
- vector_stores

## 📊 Initial Data Setup

### 1. Create Admin User

```bash
cd server
node -e "
const { User } = require('./models');
(async () => {
  try {
    const admin = await User.create({
      name: 'Admin User',
      email: 'admin@yourdomain.com',
      preferences: { role: 'admin' },
      isActive: true
    });
    console.log('✅ Admin user created:', admin.email);
  } catch(e) {
    console.error('❌ Error:', e.message);
  }
})();
"
```

### 2. Load Default Assistants (Optional)

If you have the assistant data file:

```bash
cd server
node -e "
const fs = require('fs');
const { Assistant } = require('./models');
const assistantsData = JSON.parse(fs.readFileSync('./data/assistants.json'));
(async () => {
  try {
    await Assistant.bulkCreate(assistantsData);
    console.log('✅ Default assistants loaded');
  } catch(e) {
    console.error('❌ Error:', e.message);
  }
})();
"
```

### 3. Verify Data

```bash
psql -h localhost -U ideas_gen_user -d ideas_gen_dev -c "SELECT COUNT(*) FROM assistants;"
psql -h localhost -U ideas_gen_user -d ideas_gen_dev -c "SELECT COUNT(*) FROM users;"
```

## 🏃‍♂️ Running the Application

### Development Mode

### 1. Start Backend Server

```bash
cd server
npm run dev
# or
nodemon index.js
```

Server should start on http://localhost:3000

### 2. Start Frontend Development Server

```bash
cd admin
npm run dev
```

Frontend should start on http://localhost:5173

### 3. Verify Installation

Visit http://localhost:5173 and check:
- ✅ Login page loads
- ✅ Can log in with admin credentials
- ✅ Home page shows assistants
- ✅ Chat interface works
- ✅ Admin dashboard accessible

### Production Mode

### 1. Build Frontend

```bash
cd admin
npm run build
```

### 2. Start Backend with PM2

```bash
cd server
pm2 start ecosystem.config.js --env production
```

Create `server/ecosystem.config.js`:

```javascript
module.exports = {
  apps: [{
    name: 'ideas-gen-2025',
    script: 'index.js',
    cwd: '/path/to/your/server',
    instances: 'max',
    exec_mode: 'cluster',
    env: {
      NODE_ENV: 'development'
    },
    env_production: {
      NODE_ENV: 'production',
      PORT: 3000
    },
    error_file: '/var/log/ideas-gen/error.log',
    out_file: '/var/log/ideas-gen/out.log',
    log_file: '/var/log/ideas-gen/combined.log',
    time: true
  }]
}
```

## 🌐 Production Deployment

### Using Docker (Recommended)

### 1. Create Docker Files

Create `Dockerfile`:

```dockerfile
# Multi-stage build
FROM node:18-alpine AS frontend-build
WORKDIR /app/admin
COPY admin/package*.json ./
RUN npm ci --only=production
COPY admin/ ./
RUN npm run build

FROM node:18-alpine AS backend
WORKDIR /app
COPY server/package*.json ./
RUN npm ci --only=production
COPY server/ ./
COPY --from=frontend-build /app/admin/dist ./public

EXPOSE 3000
CMD ["npm", "start"]
```

Create `docker-compose.yml`:

```yaml
version: '3.8'

services:
  app:
    build: .
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
      - DB_HOST=postgres
      - DB_NAME=ideas_gen_prod
      - DB_USER=ideas_gen_user
      - DB_PASSWORD=${DB_PASSWORD}
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - JWT_SECRET=${JWT_SECRET}
    depends_on:
      - postgres
    volumes:
      - ./uploads:/app/uploads

  postgres:
    image: postgres:14-alpine
    environment:
      - POSTGRES_DB=ideas_gen_prod
      - POSTGRES_USER=ideas_gen_user
      - POSTGRES_PASSWORD=${DB_PASSWORD}
    volumes:
      - postgres_data:/var/lib/postgresql/data
    ports:
      - "5432:5432"

  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./ssl:/etc/ssl/certs
    depends_on:
      - app

volumes:
  postgres_data:
```

### 2. Deploy

```bash
# Create environment file
echo "DB_PASSWORD=your_secure_password" > .env
echo "OPENAI_API_KEY=your_openai_key" >> .env
echo "JWT_SECRET=your_jwt_secret" >> .env

# Deploy
docker-compose up -d --build
```

### Using Traditional Server

### 1. Setup Nginx

Create `/etc/nginx/sites-available/ideas-gen-2025`:

```nginx
server {
    listen 80;
    server_name your-domain.com;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name your-domain.com;

    ssl_certificate /path/to/your/certificate.crt;
    ssl_certificate_key /path/to/your/private.key;

    # Frontend static files
    location / {
        root /var/www/ideas-gen-2025/admin/dist;
        try_files $uri $uri/ /index.html;
    }

    # API proxy
    location /api {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
    }
}
```

### 2. Setup SSL (Let's Encrypt)

```bash
sudo apt install certbot python3-certbot-nginx
sudo certbot --nginx -d your-domain.com
```

### 3. Setup Process Manager

```bash
cd server
pm2 start index.js --name "ideas-gen-2025" --env production
pm2 save
pm2 startup
```

## 🔍 Troubleshooting

### Common Issues

#### 1. Database Connection Failed

**Error**: `ECONNREFUSED` or `database does not exist`

**Solutions**:
```bash
# Check PostgreSQL status
sudo systemctl status postgresql

# Restart PostgreSQL
sudo systemctl restart postgresql

# Verify database exists
psql -l | grep ideas_gen

# Recreate database if needed
createdb -U ideas_gen_user ideas_gen_dev
```

#### 2. OpenAI API Errors

**Error**: `Invalid API key` or `Rate limit exceeded`

**Solutions**:
- Verify API key in `.env` file
- Check OpenAI account billing and limits
- Test API key:
```bash
curl -H "Authorization: Bearer $OPENAI_API_KEY" \
     https://api.openai.com/v1/models
```

#### 3. File Upload Issues

**Error**: `ENOENT: no such file or directory`

**Solutions**:
```bash
# Create uploads directory
mkdir -p server/uploads
chmod 755 server/uploads

# Check file size limits in .env
MAX_FILE_SIZE=10485760  # 10MB
```

#### 4. Frontend Build Errors

**Error**: `Module not found` or build failures

**Solutions**:
```bash
# Clear cache and reinstall
rm -rf admin/node_modules
rm admin/package-lock.json
cd admin && npm install

# Clear Vite cache
rm -rf admin/.vite
```

#### 5. CORS Issues

**Error**: `blocked by CORS policy`

**Solutions**:
- Check `CORS_ORIGIN` in server `.env`
- Verify frontend URL matches CORS setting
- For development, use: `CORS_ORIGIN=http://localhost:5173`

### Debug Commands

```bash
# Check server logs
cd server && npm run dev

# Test database connection
node -e "const {sequelize} = require('./models'); sequelize.authenticate().then(() => console.log('✅ DB OK')).catch(console.error);"

# Test API endpoints
curl http://localhost:3000/health
curl http://localhost:3000/api/assistants

# Check frontend build
cd admin && npm run build

# Test production build locally
cd admin && npm run preview
```

## 🔧 Maintenance

### Regular Tasks

#### 1. Database Backups

```bash
# Create backup script
#!/bin/bash
DATE=$(date +%Y%m%d_%H%M%S)
pg_dump -h localhost -U ideas_gen_user ideas_gen_prod > backup_$DATE.sql

# Automate with cron
echo "0 2 * * * /path/to/backup_script.sh" | crontab -
```

#### 2. Log Rotation

```bash
# Setup logrotate for PM2
sudo nano /etc/logrotate.d/pm2

/home/user/.pm2/logs/*.log {
    daily
    missingok
    rotate 7
    compress
    notifempty
    create 0644 user user
    postrotate
        pm2 reloadLogs
    endscript
}
```

#### 3. Security Updates

```bash
# Update dependencies monthly
cd server && npm audit fix
cd admin && npm audit fix

# Update system packages
sudo apt update && sudo apt upgrade
```

#### 4. Performance Monitoring

```bash
# Monitor with PM2
pm2 monit

# Check system resources
htop
df -h
free -h

# Monitor database
psql -c "SELECT * FROM pg_stat_activity;"
```

### Scaling Considerations

#### 1. Database Scaling
- Consider PostgreSQL read replicas for heavy read workloads
- Implement connection pooling with `pg-pool`
- Monitor query performance and add indexes

#### 2. Application Scaling
- Use PM2 cluster mode: `pm2 start index.js -i max`
- Implement Redis for session storage
- Consider load balancer for multiple servers

#### 3. Storage Scaling
- Use cloud storage (AWS S3, Google Cloud Storage) for file uploads
- Implement CDN for static assets
- Regular cleanup of old conversation data

---

## 📞 Support

For issues and questions:

1. Check this installation guide
2. Review troubleshooting section
3. Check application logs
4. Contact system administrator

---

**Installation Guide Version**: 2.0.0  
**Last Updated**: September 2025  
**Compatible with**: Ideas Generator 2025 v2.x