Manish Tanwar ac275a9259 inter-project-deployment-updates

2026-02-27 20:33:26 +05:30

21 KiB

Raw Blame History

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Lux Studio is an AI-powered cinematography suite for professional image and video generation, combining physics-based prompt engineering with Google's Imagen 3 and Veo 3.1 APIs. The application uses a project-first workflow with local IndexedDB storage.

Directory Structure

cinema-studio-pro/
├── frontend/              # React frontend application
│   ├── src/
│   │   ├── components/    # React components
│   │   ├── hooks/         # Custom hooks (IndexedDB, projects)
│   │   ├── authConfig.js  # MSAL authentication config
│   │   └── App.jsx        # Root component
│   ├── .env               # Frontend environment variables (gitignored)
│   ├── .env.local         # Local dev template
│   ├── .env.production    # Production template
│   └── package.json
├── backend/               # PHP backend APIs
│   ├── api.php            # Image generation API
│   ├── video_api.php      # Video generation API
│   ├── stream_video.php   # Video streaming
│   ├── enhance_prompt.php # Prompt optimization
│   ├── session_manager.php # Multi-user session management
│   ├── AuthMiddleware.php # SSO authentication (required by api.php)
│   ├── JWTValidator.php   # Azure AD token validation
│   ├── uploads/           # Temporary file storage
│   ├── .env               # Backend environment variables (gitignored)
│   ├── .env.local         # Local dev template
│   ├── .env.production    # Production template
│   └── composer.json      # PHP dependencies
├── README.md              # User-facing project overview
├── NEW_DEPLOYMENT.md      # Production deployment guide (FileZilla + SSH)
└── CLAUDE.md              # This file

Development Commands

Quick Setup (Recommended)

./setup.sh                     # Automated setup for local development
                               # Auto-detects port conflicts, kills stale processes
                               # Generates stop.sh and starts both servers
./stop.sh                      # Stop servers started by setup.sh (auto-generated)
./status.sh                    # Check if servers are running + last 3 log lines

Frontend (React + Vite)

cd frontend
npm install                    # Install dependencies
npm run dev                    # Start dev server (port from .env, default: 3000)
npm run build                  # Build for production
npm run preview                # Preview production build
npm run lint                   # Run ESLint

Backend (PHP)

cd backend
composer install               # Install PHP dependencies (Firebase JWT)
# Port is configured in .env (BACKEND_PORT, default: 5015)
php -S localhost:5015          # Start PHP development server

Full Development Setup

# Terminal 1: Backend (reads port from backend/.env)
cd backend && php -S localhost:5015

# Terminal 2: Frontend (reads port from frontend/.env)
cd frontend && npm run dev

# Visit: http://localhost:3000 (or your configured FRONTEND_PORT)

Note: Ports are configured via .env files. See Configuration section below.

Architecture Overview

Two-Part System

Frontend (frontend/):

React 19 + Vite + Tailwind CSS
MSAL authentication with Azure AD (toggleable)
IndexedDB for local project/media storage via useProjects.js and useIndexedDB.js hooks
Three main tabs: Projects, Image Gen, Video Gen
No server-side session storage - all state in browser
Runs on port 3000 (configurable via FRONTEND_PORT in .env)

Backend (backend/):

PHP 7.4+ stateless APIs
Session-based user isolation (multi-user support via SessionManager)
Direct API calls to Google's Gemini/Imagen/Veo services
File-based storage in backend/uploads/sessions/{session_id}/
Runs on port 5015 (configurable via BACKEND_PORT in .env)

Data Flow Architecture

Project-First Workflow:

User creates/selects project in ProjectsTab (stored in IndexedDB)
Image Gen and Video Gen tabs are DISABLED until project selected
All generations save to active project automatically
Projects contain: images, videos, metadata, storyboards

Image Generation (Imagen 3):

CinePromptStudio → /api/api.php (proxied to backend:5015) → Gemini API (Imagen 3) →
Base64 response → Server stores in backend/uploads/sessions/{id}/ → Frontend saves to IndexedDB project

Video Generation (Veo 3.1):

VideoGenTab → /api/video_api.php (proxied to backend:5015) → Gemini API (Veo 3.1) →
Operation ID → Async polling → Video base64 → Server stores → Frontend saves to project

Authentication Flow (MSAL SSO):

User visits localhost:3000 → LoginPage component → Azure AD popup login →
Token validated → App.jsx renders main UI → Logout button visible in header

Critical Backend Pattern: Imagen 3 Request Structure

The Imagen 3 API via Gemini has a VERY SPECIFIC request format that differs from standard image APIs:

For editing (with input image):

Image data MUST come BEFORE text prompt in the parts array
Use inline_data (snake_case) in request, but response uses inlineData (camelCase)
MIME type must match (typically image/jpeg)
Base64 must be clean (no whitespace, no data URI prefix)

See the Critical Patterns section below for the full request format rules.

Session Management (Multi-User)

The SessionManager class (session_manager.php) provides user isolation:

Each user gets unique session ID via PHP session
Files stored in uploads/sessions/{session_id}/
Images stored with timestamps and metadata
Automatic cleanup of files older than 24 hours
Session data includes: current_image, conversation_history, image_history

IndexedDB Schema (Frontend)

Database lux-studio-projects (version 3):

Store: projects - Project metadata (indexes: name, updatedAt, userId)
Store: items - All media (images AND videos) with projectId and type indexes
Store: storyboards - Storyboard annotations (indexes: projectId, updatedAt)

Note: Media is in a single items store with a type field - NOT separate images/videos stores. Schema changes require a version bump in useIndexedDB.js.

useProjects.js auto-migrates on load: adds userId: 'local' to old projects and rewrites old /stream_video.php?file= URLs to /generated_videos/ format. All managed by useProjects.js hook - never use useIndexedDB directly from components.

Key Components and Their Roles

Frontend Components

App.jsx / AppContent.jsx - App.jsx is a thin wrapper that renders <AppContent /> inside MsalProvider. All MSAL hooks (useIsAuthenticated, useMsal) live in AppContent - using them directly in App.jsx would break because hooks can't be called outside their provider. AppContent manages:

Active tab state (projects/image/video)
Active project selection (blocks Image/Video Gen when null)
Video rerun data flow (from ProjectsTab to VideoGenTab)
Image edit data flow (from ProjectsTab to CinePromptStudio)

ProjectsTab.jsx - Project management:

Create/select/delete projects
View project image/video libraries
Trigger video rerun with original params
Trigger image edit in Image Gen tab
Storyboard editor integration

CinePromptStudio.jsx - Image generation:

40+ lighting presets, 8 camera bodies, 10 lens profiles
Reference images (up to 14 via Imagen 3)
Edit mode - refines last generated image
Image upload for style transfer or editing
Saves to active project via useProjects hook

VideoGenTab.jsx - Video generation:

Text-to-video and Image-to-video modes
Reference images (up to 2: first frame + optional last frame for interpolation)
Veo 3.1 Standard or Fast model selection
Creative Freedom levels (auto-adjusts guidance strength)
Duration options: 4s, 6s, 8s (all available for both T2V and I2V)
Async polling for video completion
Frame extraction from generated videos

VideoPlayer.jsx - Video playback:

Video streaming via stream_video.php with Range header support
Frame extraction - captures frame as PNG, adds to project images
Used in VideoGenTab and ProjectsTab

StoryboardEditor.jsx - Storyboard annotations:

Drag-to-reorder panels (@dnd-kit)
Add annotations to images/videos
Export to PDF (html2canvas + jspdf)

Backend APIs

api.php - Image generation:

NanoBananaProAPI class handles Imagen 3 requests
SessionManager for user isolation
Handles input image (editing), reference images (style)
Returns success with metadata or error details
Critical: Image must come BEFORE text in parts array

video_api.php - Video generation:

VeoVideoAPI class handles Veo 3.1 requests
Two models: 'standard' (veo-3.1-generate-preview) or 'fast' (veo-3.1-fast-generate-preview)
Async operations - returns operation ID immediately
Stores operations in video_operations.json for polling
Reference images: first is starting frame, optional second is ending frame (interpolation)

stream_video.php - Video streaming:

HTTP Range header support for seeking
Used by VideoPlayer for playback

enhance_prompt.php - Prompt optimization:

Uses Gemini Pro for prompt enhancement
Camera equipment presets inject physics-based characteristics
Video prompts get AI-inferred camera movement and subject actions in brackets

webhook_logger.php - Operation tracking:

Logs all Gemini API requests/responses
Tracks async video operations
Useful for debugging API issues

Authentication (Optional)

AuthMiddleware.php - SSO orchestration:

Toggleable via SSO_ENABLED in .env
When false: returns mock "Local Developer" user
When true: validates Azure AD JWT tokens
Used by api.php and enhance_prompt.php

JWTValidator.php - Token validation:

Validates Azure AD ID tokens
Checks expiration, audience, issuer
Uses Azure JWKS for signature verification

Important Configuration Files

Environment Configuration Strategy

The application uses .env files as the single source of truth for all configuration:

Local Development: setup.sh copies from .env.local files (pre-configured for localhost)
Production: deploy.sh copies from .env.production files (pre-configured for production)
Port Control: Both frontend and backend ports are configurable via env files
Current .env: Working .env files are gitignored and created by setup/deploy scripts

Backend Configuration

backend/.env.local - Local development template (copied by setup.sh):

BACKEND_PORT=5015
GEMINI_API_KEY=AIzaSyC...
FRONTEND_URL=http://localhost:3000
SSO_ENABLED=false
SSO_TENANT_ID=e519c2e6-bc6d-4fdf-8d9c-923c2f002385
SSO_CLIENT_ID=15c0c4e2-bac0-4564-a3a6-c2717f00a6d9

backend/.env.production - Production template (copied by deploy.sh):

BACKEND_PORT=5015
GEMINI_API_KEY=AIzaSyC...
FRONTEND_URL=https://ai-sandbox.oliver.solutions/lux-studio
SSO_ENABLED=false
SSO_TENANT_ID=e519c2e6-bc6d-4fdf-8d9c-923c2f002385
SSO_CLIENT_ID=9079054c-9620-4757-a256-23413042f1ef

backend/.env - Working config (gitignored, created by setup.sh or deploy.sh)

Frontend Configuration

frontend/.env.local - Local development template (copied by setup.sh):

FRONTEND_PORT=3000
BACKEND_PORT=5015
VITE_BASE_PATH=/
VITE_API_URL=http://localhost:5015
VITE_GEMINI_API_KEY=AIzaSyC...
VITE_SSO_ENABLED=true
VITE_SSO_TENANT_ID=e519c2e6-bc6d-4fdf-8d9c-923c2f002385
VITE_SSO_CLIENT_ID=15c0c4e2-bac0-4564-a3a6-c2717f00a6d9
VITE_SSO_REDIRECT_URI=http://localhost:3000
NODE_ENV=development

frontend/.env.production - Production template (copied by deploy.sh):

# NOTE: Port variables NOT used in production (only for local dev)
# Production uses Apache on port 443 to serve built static files
VITE_BASE_PATH=/lux-studio/
VITE_API_URL=https://ai-sandbox.oliver.solutions/lux-studio/api
VITE_GEMINI_API_KEY=AIzaSyC...
VITE_SSO_ENABLED=true
VITE_SSO_TENANT_ID=e519c2e6-bc6d-4fdf-8d9c-923c2f002385
VITE_SSO_CLIENT_ID=9079054c-9620-4757-a256-23413042f1ef
VITE_SSO_REDIRECT_URI=https://ai-sandbox.oliver.solutions/lux-studio/
NODE_ENV=production

frontend/.env - Working config (gitignored, created by setup.sh or deploy.sh)

Configuration Notes

Local Development: Run ./setup.sh which copies .env.local → .env for both frontend and backend
Production Deployment: Run sudo ./deploy.sh which copies .env.production → .env for both frontend and backend
Port Changes: Modify ports in .env.local (for local) or .env.production (for production)
SSO Toggle: Set VITE_SSO_ENABLED=false in the appropriate env file to disable authentication
Vite Config: vite.config.js automatically reads configuration from .env files

backend/composer.json - PHP dependencies:

firebase/php-jwt - JWT validation for Azure AD tokens

frontend/package.json - Frontend dependencies:

@azure/msal-browser + @azure/msal-react - Microsoft Authentication Library
@google/generative-ai - Direct Gemini JS client (used in VideoGenTab for prompt enhancement)
@dnd-kit/core + @dnd-kit/sortable - Storyboard drag-to-reorder
html2canvas + jspdf - Storyboard PDF export
lucide-react - Icon library

Common Development Tasks

Adding a New Camera Profile

Edit CinePromptStudio.jsx - add to cameraProfiles array with characteristics like sensor behavior, dynamic range, grain.

Adding a New Lens Profile

Edit CinePromptStudio.jsx - add to lensProfiles array with optical traits like bokeh, flares, aberrations.

Modifying Video Duration Options

Edit VideoGenTab.jsx - update durationOptions array, ensure constraints match Veo 3.1 API limits (4/6/8s).

Debugging Image Generation Issues

Check API URL configuration - Verify VITE_API_URL in frontend/.env matches backend server
Check browser console for API errors (404/500 = wrong API URL)
Check PHP error_log in project root
Verify image editing parts order: image data MUST precede text (see Critical Patterns section)
Check finishReason in API response (IMAGE_RECITATION = blocked)
Ensure prompts are creative/detailed (10+ words)

Debugging Video Generation Issues

Check API URL configuration - Verify VITE_API_URL in frontend/.env matches backend server
Check video_operations.json for operation status
Check PHP error_log for API errors
Verify webhook logs in webhook_logger.php calls
Check async polling interval in VideoGenTab (5 seconds default)
Ensure reference images are base64 without data URI prefix

Critical Patterns to Follow

1. API URL Configuration

IMPORTANT: All API calls MUST use the getApiUrl() helper function to avoid CORS issues:

// At the top of component (after state hooks)
const getApiUrl = (endpoint) => {
  // In development, use Vite proxy to avoid CORS
  if (import.meta.env.DEV) {
    return `/api/${endpoint}`;
  }
  // In production, use full API URL
  const apiUrl = import.meta.env.VITE_API_URL || 'http://localhost:5015';
  return `${apiUrl}/${endpoint}`;
};

// In fetch calls
const response = await fetch(getApiUrl('api.php'), { ... });

NEVER hardcode API paths like /lux-studio-back/api.php or use direct VITE_API_URL in development.

Why this pattern:

Development mode (import.meta.env.DEV === true): Uses Vite's proxy (/api/) which automatically forwards to backend without CORS issues
Production build: Uses full VITE_API_URL for direct API calls
vite.config.js proxies to backend: /api/*, /generated_videos/*, /generated_images/* → http://localhost:{BACKEND_PORT}/*

2. Image Editing Request Order

When editing images, input image MUST come before text in parts array:

parts: [
  { inline_data: { mime_type: 'image/jpeg', data: base64 } },  // First!
  { text: 'edit instruction' }                                  // Second!
]

3. Base64 Data Cleaning

Always strip data URI prefix before sending to backend:

const base64 = reader.result.split(',')[1];  // Remove "data:image/jpeg;base64,"

4. Project Selection Enforcement

Image and Video Gen tabs check for activeProjectId - never allow generation without project:

if ((tabId === 'image' || tabId === 'video') && !activeProjectId) {
  return; // Block tab change
}

5. Video Generation Settings

All durations (4s, 6s, 8s) work for both T2V and I2V modes. Resolution (720p/1080p) is a UI-only setting - only aspectRatio and durationSeconds are sent to the Veo 3.1 API.

6. MIME Type Handling

Always store and use dynamic MIME type from API response:

$_SESSION['current_image'] = $base64;
$_SESSION['current_image_mime'] = $mimeType;  // Don't hardcode 'image/png'

7. IndexedDB Project Operations

Always use the useProjects hook for database operations - don't use useIndexedDB directly from components.

8. ESLint Unused Variable Rule

eslint.config.js uses varsIgnorePattern: '^[A-Z_]' - uppercase-named vars (constants, React components) won't trigger no-unused-vars. This is intentional.

9. Veo Last Frame Interpolation Constraint

The lastFrame reference image (second image in I2V mode) is only applicable when duration is 8s. The feature may not be fully available in the Gemini API yet - check video_api.php comments.

Error Handling Patterns

Imagen 3 Errors

IMAGE_RECITATION - Prompt too generic/simple, needs creative detail
SAFETY - Content filter triggered, try different prompt
finishReason !== 'STOP' - Generation failed for reason specified

Veo 3.1 Errors

Operation stuck in PENDING - Check webhook logs, may need retry
Operation FAILED - Check error message in operation response
Timeout after 5 minutes polling - Display timeout error to user

Session/Storage Errors

Session files not saving - Check uploads/sessions/ permissions (755)
IndexedDB quota exceeded - Prompt user to delete old projects
CORS errors - Ensure PHP backend allows origin

Testing Checklist

Before committing changes:

Test with no project selected (Image/Video Gen should be disabled)
Test image generation with creative prompt (10+ words)
Test image editing (uses last generated image)
Test video generation T2V and I2V modes
Test video rerun from project library
Test frame extraction from video
Test storyboard drag-to-reorder
Test project deletion (cleans up IndexedDB)
Test with SSO_ENABLED=false (mock auth)
Check PHP error_log for any errors
Verify generated files land in correct session directory

Authentication

The application uses MSAL (Microsoft Authentication Library) for Azure AD SSO:

Enabled by default in development with provided credentials
Toggleable via VITE_SSO_ENABLED in frontend/.env
LoginPage component handles authentication UI
App.jsx checks authentication status and shows login or main app
Logout button in header when authenticated (top-right corner)

SSO Flow:

User visits localhost:3000
If not authenticated, LoginPage is shown
User clicks "Sign in with Microsoft"
MSAL popup opens for Azure AD login
After successful login, main app renders
User info and logout button appear in header

Documentation References

README.md - User-facing project overview, setup, and troubleshooting
NEW_DEPLOYMENT.md - Production deployment guide (FileZilla + SSH)

API Rate Limits

Google Gemini APIs:

Rate limit: Handled by exponential backoff in API calls
HTTP 429 errors: Retry after 30 seconds
HTTP 500 errors: Temporary API issue, retry with backoff

File Storage Patterns

Backend (temporary, auto-cleanup after 24h):

backend/uploads/sessions/{session_id}/images/*.jpg - Generated images
backend/uploads/sessions/{session_id}/videos/*.mp4 - Generated videos
backend/video_operations.json - Pending video operations

Frontend (persistent, user-managed):

IndexedDB lux-studio-projects database
Projects, images, videos, storyboards stores
No automatic cleanup - user deletes projects manually

Build and Deployment

Frontend build:

cd frontend
cp .env.production .env
npm run build
# Outputs to frontend/dist/
# Verify: grep -o "lux-studio/api" frontend/dist/assets/*.js | head -3

Deployment (FileZilla + SSH — see NEW_DEPLOYMENT.md for the full guide):

Server: ai-sandbox.oliver.solutions
Frontend static files → /var/www/html/lux-studio/
Backend PHP files → /var/www/html/lux-studio/api/
Delete old assets/ folder on server before uploading new one (hash-named files change each build)
Never overwrite the server's /api/.env (contains production secrets)
AuthMiddleware.php is REQUIRED in /api/ — api.php will crash without it

Production checklist:

PHP 7.4+ installed on server
backend/.env on server with valid GEMINI_API_KEY and SSO_ENABLED=false
backend/uploads/sessions/ writable by web server (chmod 755)
HTTPS enabled (required for SSO redirect URI)
Frontend built with VITE_API_URL=https://ai-sandbox.oliver.solutions/lux-studio/api
CORS configured on backend for frontend domain (FRONTEND_URL in server .env)

21 KiB Raw Blame History