diff --git a/INSTALLATION.md b/INSTALLATION.md new file mode 100644 index 0000000..ba1dd95 --- /dev/null +++ b/INSTALLATION.md @@ -0,0 +1,744 @@ +# Ideas Generator 2025 - Installation Guide + +A comprehensive installation guide for deploying the Ideas Generator 2025 application with dual-agent system, file upload support, and complete admin dashboard. + +## 📋 Table of Contents + +1. [Prerequisites](#prerequisites) +2. [Environment Setup](#environment-setup) +3. [Database Configuration](#database-configuration) +4. [Backend Installation](#backend-installation) +5. [Frontend Installation](#frontend-installation) +6. [Environment Variables](#environment-variables) +7. [Database Migration](#database-migration) +8. [Initial Data Setup](#initial-data-setup) +9. [Running the Application](#running-the-application) +10. [Production Deployment](#production-deployment) +11. [Troubleshooting](#troubleshooting) +12. [Maintenance](#maintenance) + +## 🔧 Prerequisites + +### System Requirements +- **Node.js**: 18.0.0 or higher +- **npm**: 9.0.0 or higher +- **PostgreSQL**: 14.0 or higher +- **Git**: Latest version + +### Required Services +- **OpenAI API Account** with access to: + - Chat Completions API + - Responses API + - File Upload API + - Vector Stores (optional) + +### Development Tools (Optional) +- **pgAdmin** or similar PostgreSQL management tool +- **Postman** for API testing +- **VS Code** with recommended extensions + +## 🌍 Environment Setup + +### 1. Clone the Repository + +```bash +git clone https://bitbucket.org/zlalani/ideas-generator.git +cd ideas-generator +git checkout ideas-gen-2025 +``` + +### 2. Verify Node.js Installation + +```bash +node --version # Should be 18.0.0+ +npm --version # Should be 9.0.0+ +``` + +### 3. Install Global Dependencies + +```bash +npm install -g nodemon +npm install -g pm2 # For production deployment +``` + +## 🗄️ Database Configuration + +### 1. Install PostgreSQL + +**macOS (using Homebrew):** +```bash +brew install postgresql@14 +brew services start postgresql@14 +``` + +**Ubuntu/Debian:** +```bash +sudo apt update +sudo apt install postgresql postgresql-contrib +sudo systemctl start postgresql +sudo systemctl enable postgresql +``` + +**Windows:** +Download and install from [PostgreSQL Official Site](https://www.postgresql.org/download/windows/) + +### 2. Create Database and User + +```bash +# Connect to PostgreSQL as superuser +sudo -u postgres psql + +# Create database and user +CREATE DATABASE ideas_gen_dev; +CREATE USER ideas_gen_user WITH ENCRYPTED PASSWORD 'your_secure_password'; +GRANT ALL PRIVILEGES ON DATABASE ideas_gen_dev TO ideas_gen_user; + +# For production, create separate database +CREATE DATABASE ideas_gen_prod; +GRANT ALL PRIVILEGES ON DATABASE ideas_gen_prod TO ideas_gen_user; + +\q +``` + +### 3. Test Database Connection + +```bash +psql -h localhost -U ideas_gen_user -d ideas_gen_dev +``` + +## 🚀 Backend Installation + +### 1. Navigate to Server Directory + +```bash +cd server +``` + +### 2. Install Dependencies + +```bash +npm install +``` + +### 3. Install Additional Dependencies (if needed) + +```bash +npm install --save sequelize pg pg-hstore +npm install --save express cors helmet morgan +npm install --save openai multer node-fetch +npm install --save-dev nodemon +``` + +## 🎨 Frontend Installation + +### 1. Navigate to Admin Directory + +```bash +cd ../admin +``` + +### 2. Install Dependencies + +```bash +npm install +``` + +### 3. Install Additional Dependencies (if needed) + +```bash +npm install --save vue@next vue-router@4 +npm install --save axios chart.js +npm install --save marked +npm install --save-dev @vitejs/plugin-vue vite +``` + +## 🔐 Environment Variables + +### 1. Server Environment (.env) + +Create `server/.env` file: + +```env +# Application Settings +NODE_ENV=development +PORT=3000 +FRONTEND_URL=http://localhost:5173 + +# Database Configuration +DB_NAME=ideas_gen_dev +DB_USER=ideas_gen_user +DB_PASSWORD=your_secure_password +DB_HOST=localhost +DB_PORT=5432 +DB_DIALECT=postgres + +# OpenAI API Configuration +OPENAI_API_KEY=sk-your-openai-api-key-here +OPENAI_ORG_ID=org-your-organization-id-here + +# Security Settings +JWT_SECRET=your-super-secure-jwt-secret-key-here +SESSION_SECRET=your-super-secure-session-secret-here + +# File Upload Settings +MAX_FILE_SIZE=10485760 +UPLOAD_PATH=./uploads + +# CORS Settings +CORS_ORIGIN=http://localhost:5173 + +# Logging +LOG_LEVEL=debug +``` + +### 2. Frontend Environment (.env) + +Create `admin/.env` file: + +```env +# API Configuration +VITE_API_BASE_URL=http://localhost:3000 +VITE_API_TIMEOUT=30000 + +# Application Settings +VITE_APP_NAME=Ideas Generator 2025 +VITE_APP_VERSION=2.0.0 + +# Development Settings +VITE_DEV_TOOLS=true +VITE_LOG_LEVEL=debug +``` + +### 3. Production Environment Variables + +For production, create `server/.env.production`: + +```env +# Application Settings +NODE_ENV=production +PORT=3000 +FRONTEND_URL=https://your-domain.com + +# Database Configuration +DB_NAME=ideas_gen_prod +DB_USER=ideas_gen_user +DB_PASSWORD=your_production_password +DB_HOST=your-db-host +DB_PORT=5432 +DB_DIALECT=postgres +DB_SSL=true + +# OpenAI API Configuration +OPENAI_API_KEY=sk-your-production-openai-key +OPENAI_ORG_ID=org-your-production-org-id + +# Security Settings (Generate strong secrets!) +JWT_SECRET=your-production-jwt-secret +SESSION_SECRET=your-production-session-secret + +# File Upload Settings +MAX_FILE_SIZE=10485760 +UPLOAD_PATH=/var/www/uploads + +# CORS Settings +CORS_ORIGIN=https://your-domain.com + +# Logging +LOG_LEVEL=info +``` + +## 🗄️ Database Migration + +### 1. Run Database Migrations + +```bash +cd server +npm run migrate +``` + +### 2. If migrations don't exist, sync models + +```bash +cd server +node -e " +const { sequelize } = require('./models'); +sequelize.sync({ force: false }) + .then(() => console.log('✅ Database synced')) + .catch(err => console.error('❌ Sync failed:', err)); +" +``` + +### 3. Verify Tables Created + +```bash +psql -h localhost -U ideas_gen_user -d ideas_gen_dev -c "\dt" +``` + +Expected tables: +- users +- assistants +- conversations +- messages +- response_sessions +- vector_stores + +## 📊 Initial Data Setup + +### 1. Create Admin User + +```bash +cd server +node -e " +const { User } = require('./models'); +(async () => { + try { + const admin = await User.create({ + name: 'Admin User', + email: 'admin@yourdomain.com', + preferences: { role: 'admin' }, + isActive: true + }); + console.log('✅ Admin user created:', admin.email); + } catch(e) { + console.error('❌ Error:', e.message); + } +})(); +" +``` + +### 2. Load Default Assistants (Optional) + +If you have the assistant data file: + +```bash +cd server +node -e " +const fs = require('fs'); +const { Assistant } = require('./models'); +const assistantsData = JSON.parse(fs.readFileSync('./data/assistants.json')); +(async () => { + try { + await Assistant.bulkCreate(assistantsData); + console.log('✅ Default assistants loaded'); + } catch(e) { + console.error('❌ Error:', e.message); + } +})(); +" +``` + +### 3. Verify Data + +```bash +psql -h localhost -U ideas_gen_user -d ideas_gen_dev -c "SELECT COUNT(*) FROM assistants;" +psql -h localhost -U ideas_gen_user -d ideas_gen_dev -c "SELECT COUNT(*) FROM users;" +``` + +## 🏃‍♂️ Running the Application + +### Development Mode + +### 1. Start Backend Server + +```bash +cd server +npm run dev +# or +nodemon index.js +``` + +Server should start on http://localhost:3000 + +### 2. Start Frontend Development Server + +```bash +cd admin +npm run dev +``` + +Frontend should start on http://localhost:5173 + +### 3. Verify Installation + +Visit http://localhost:5173 and check: +- ✅ Login page loads +- ✅ Can log in with admin credentials +- ✅ Home page shows assistants +- ✅ Chat interface works +- ✅ Admin dashboard accessible + +### Production Mode + +### 1. Build Frontend + +```bash +cd admin +npm run build +``` + +### 2. Start Backend with PM2 + +```bash +cd server +pm2 start ecosystem.config.js --env production +``` + +Create `server/ecosystem.config.js`: + +```javascript +module.exports = { + apps: [{ + name: 'ideas-gen-2025', + script: 'index.js', + cwd: '/path/to/your/server', + instances: 'max', + exec_mode: 'cluster', + env: { + NODE_ENV: 'development' + }, + env_production: { + NODE_ENV: 'production', + PORT: 3000 + }, + error_file: '/var/log/ideas-gen/error.log', + out_file: '/var/log/ideas-gen/out.log', + log_file: '/var/log/ideas-gen/combined.log', + time: true + }] +} +``` + +## 🌐 Production Deployment + +### Using Docker (Recommended) + +### 1. Create Docker Files + +Create `Dockerfile`: + +```dockerfile +# Multi-stage build +FROM node:18-alpine AS frontend-build +WORKDIR /app/admin +COPY admin/package*.json ./ +RUN npm ci --only=production +COPY admin/ ./ +RUN npm run build + +FROM node:18-alpine AS backend +WORKDIR /app +COPY server/package*.json ./ +RUN npm ci --only=production +COPY server/ ./ +COPY --from=frontend-build /app/admin/dist ./public + +EXPOSE 3000 +CMD ["npm", "start"] +``` + +Create `docker-compose.yml`: + +```yaml +version: '3.8' + +services: + app: + build: . + ports: + - "3000:3000" + environment: + - NODE_ENV=production + - DB_HOST=postgres + - DB_NAME=ideas_gen_prod + - DB_USER=ideas_gen_user + - DB_PASSWORD=${DB_PASSWORD} + - OPENAI_API_KEY=${OPENAI_API_KEY} + - JWT_SECRET=${JWT_SECRET} + depends_on: + - postgres + volumes: + - ./uploads:/app/uploads + + postgres: + image: postgres:14-alpine + environment: + - POSTGRES_DB=ideas_gen_prod + - POSTGRES_USER=ideas_gen_user + - POSTGRES_PASSWORD=${DB_PASSWORD} + volumes: + - postgres_data:/var/lib/postgresql/data + ports: + - "5432:5432" + + nginx: + image: nginx:alpine + ports: + - "80:80" + - "443:443" + volumes: + - ./nginx.conf:/etc/nginx/nginx.conf + - ./ssl:/etc/ssl/certs + depends_on: + - app + +volumes: + postgres_data: +``` + +### 2. Deploy + +```bash +# Create environment file +echo "DB_PASSWORD=your_secure_password" > .env +echo "OPENAI_API_KEY=your_openai_key" >> .env +echo "JWT_SECRET=your_jwt_secret" >> .env + +# Deploy +docker-compose up -d --build +``` + +### Using Traditional Server + +### 1. Setup Nginx + +Create `/etc/nginx/sites-available/ideas-gen-2025`: + +```nginx +server { + listen 80; + server_name your-domain.com; + return 301 https://$server_name$request_uri; +} + +server { + listen 443 ssl http2; + server_name your-domain.com; + + ssl_certificate /path/to/your/certificate.crt; + ssl_certificate_key /path/to/your/private.key; + + # Frontend static files + location / { + root /var/www/ideas-gen-2025/admin/dist; + try_files $uri $uri/ /index.html; + } + + # API proxy + location /api { + proxy_pass http://localhost:3000; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection 'upgrade'; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_cache_bypass $http_upgrade; + } +} +``` + +### 2. Setup SSL (Let's Encrypt) + +```bash +sudo apt install certbot python3-certbot-nginx +sudo certbot --nginx -d your-domain.com +``` + +### 3. Setup Process Manager + +```bash +cd server +pm2 start index.js --name "ideas-gen-2025" --env production +pm2 save +pm2 startup +``` + +## 🔍 Troubleshooting + +### Common Issues + +#### 1. Database Connection Failed + +**Error**: `ECONNREFUSED` or `database does not exist` + +**Solutions**: +```bash +# Check PostgreSQL status +sudo systemctl status postgresql + +# Restart PostgreSQL +sudo systemctl restart postgresql + +# Verify database exists +psql -l | grep ideas_gen + +# Recreate database if needed +createdb -U ideas_gen_user ideas_gen_dev +``` + +#### 2. OpenAI API Errors + +**Error**: `Invalid API key` or `Rate limit exceeded` + +**Solutions**: +- Verify API key in `.env` file +- Check OpenAI account billing and limits +- Test API key: +```bash +curl -H "Authorization: Bearer $OPENAI_API_KEY" \ + https://api.openai.com/v1/models +``` + +#### 3. File Upload Issues + +**Error**: `ENOENT: no such file or directory` + +**Solutions**: +```bash +# Create uploads directory +mkdir -p server/uploads +chmod 755 server/uploads + +# Check file size limits in .env +MAX_FILE_SIZE=10485760 # 10MB +``` + +#### 4. Frontend Build Errors + +**Error**: `Module not found` or build failures + +**Solutions**: +```bash +# Clear cache and reinstall +rm -rf admin/node_modules +rm admin/package-lock.json +cd admin && npm install + +# Clear Vite cache +rm -rf admin/.vite +``` + +#### 5. CORS Issues + +**Error**: `blocked by CORS policy` + +**Solutions**: +- Check `CORS_ORIGIN` in server `.env` +- Verify frontend URL matches CORS setting +- For development, use: `CORS_ORIGIN=http://localhost:5173` + +### Debug Commands + +```bash +# Check server logs +cd server && npm run dev + +# Test database connection +node -e "const {sequelize} = require('./models'); sequelize.authenticate().then(() => console.log('✅ DB OK')).catch(console.error);" + +# Test API endpoints +curl http://localhost:3000/health +curl http://localhost:3000/api/assistants + +# Check frontend build +cd admin && npm run build + +# Test production build locally +cd admin && npm run preview +``` + +## 🔧 Maintenance + +### Regular Tasks + +#### 1. Database Backups + +```bash +# Create backup script +#!/bin/bash +DATE=$(date +%Y%m%d_%H%M%S) +pg_dump -h localhost -U ideas_gen_user ideas_gen_prod > backup_$DATE.sql + +# Automate with cron +echo "0 2 * * * /path/to/backup_script.sh" | crontab - +``` + +#### 2. Log Rotation + +```bash +# Setup logrotate for PM2 +sudo nano /etc/logrotate.d/pm2 + +/home/user/.pm2/logs/*.log { + daily + missingok + rotate 7 + compress + notifempty + create 0644 user user + postrotate + pm2 reloadLogs + endscript +} +``` + +#### 3. Security Updates + +```bash +# Update dependencies monthly +cd server && npm audit fix +cd admin && npm audit fix + +# Update system packages +sudo apt update && sudo apt upgrade +``` + +#### 4. Performance Monitoring + +```bash +# Monitor with PM2 +pm2 monit + +# Check system resources +htop +df -h +free -h + +# Monitor database +psql -c "SELECT * FROM pg_stat_activity;" +``` + +### Scaling Considerations + +#### 1. Database Scaling +- Consider PostgreSQL read replicas for heavy read workloads +- Implement connection pooling with `pg-pool` +- Monitor query performance and add indexes + +#### 2. Application Scaling +- Use PM2 cluster mode: `pm2 start index.js -i max` +- Implement Redis for session storage +- Consider load balancer for multiple servers + +#### 3. Storage Scaling +- Use cloud storage (AWS S3, Google Cloud Storage) for file uploads +- Implement CDN for static assets +- Regular cleanup of old conversation data + +--- + +## 📞 Support + +For issues and questions: + +1. Check this installation guide +2. Review troubleshooting section +3. Check application logs +4. Contact system administrator + +--- + +**Installation Guide Version**: 2.0.0 +**Last Updated**: September 2025 +**Compatible with**: Ideas Generator 2025 v2.x \ No newline at end of file