Oliver/olivas
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Initial commit — OliVAS visual attention analysis platform

Browse source 
Full-stack application for predicting where humans look in images using
DeepGaze saliency models. Includes heatmap overlays, gaze sequence prediction,
hotspot detection, AOI analysis, rule-based insights, optional Claude AI
design analysis, and professional PDF report generation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
DJP 2026-02-23 20:20:58 -05:00
commit 3467dbcf03
113 changed files with 11778 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
.env.example Normal file
View file

@ -0,0 +1,15 @@
# Database
DATABASE_URL=postgresql+asyncpg://olivas:olivas@localhost:5453/olivas
# Storage
UPLOAD_DIR=./data/uploads
# ML Model
DEVICE=auto # auto | cpu | cuda
# CORS
CORS_ORIGINS=http://localhost:1577
# Server
BACKEND_HOST=0.0.0.0
BACKEND_PORT=8000

47
.gitignore vendored Normal file
View file

@ -0,0 +1,47 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.venv/
*.egg-info/
dist/
build/
*.egg
# Node
node_modules/
dist/
.cache/
# Environment
.env
.env.local
.env.*.local
# IDE
.vscode/
.idea/
*.swp
*.swo
*~
# OS
.DS_Store
Thumbs.db
# Data
backend/data/
backend/uploads/
*.npy
# Docker
docker-compose.override.yml
# Testing
.coverage
htmlcov/
.pytest_cache/
# Build artifacts
*.log

1
.python-version Normal file
View file

@ -0,0 +1 @@
3.12

21
LICENSE Normal file
View file

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 OLIVER
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

58
Makefile Normal file
View file

@ -0,0 +1,58 @@
.PHONY: setup setup-backend setup-frontend dev dev-backend dev-frontend db-up db-migrate test clean
# ─── Setup ───────────────────────────────────────────────────────────
setup: setup-backend setup-frontend
setup-backend:
cd backend && python3.12 -m venv .venv
cd backend && .venv/bin/pip install --upgrade pip
cd backend && .venv/bin/pip install -e ".[dev]"
cd backend && .venv/bin/pip install "deepgaze-pytorch @ git+https://github.com/matthias-k/DeepGaze.git"
setup-frontend:
cd frontend && npm install
# ─── Database ────────────────────────────────────────────────────────
db-up:
docker compose up -d postgres
db-migrate:
cd backend && .venv/bin/alembic upgrade head
db-revision:
cd backend && .venv/bin/alembic revision --autogenerate -m "$(msg)"
# ─── Development ─────────────────────────────────────────────────────
dev: db-up dev-backend dev-frontend
dev-backend:
cd backend && .venv/bin/uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
dev-frontend:
cd frontend && npm run dev
# ─── Docker ──────────────────────────────────────────────────────────
docker-up:
docker compose up --build
docker-down:
docker compose down
# ─── Testing ─────────────────────────────────────────────────────────
test:
cd backend && .venv/bin/pytest -v
test-backend:
cd backend && .venv/bin/pytest -v
# ─── Utilities ───────────────────────────────────────────────────────
lint:
cd backend && .venv/bin/ruff check app/ tests/
lint-fix:
cd backend && .venv/bin/ruff check --fix app/ tests/
clean:
find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
find . -type d -name .pytest_cache -exec rm -rf {} + 2>/dev/null || true
rm -rf backend/.venv frontend/node_modules

215
README.md Normal file
View file

@ -0,0 +1,215 @@
# OliVAS — Open-Source Visual Attention Software
**OliVAS** (OLIVER Visual Attention Suite) is an open-source web application that predicts where humans will look in an image during the first 3-5 seconds of viewing. Built for creative teams, designers, and marketers at OLIVER, it provides saliency heatmaps, gaze sequence predictions, hotspot analysis, and actionable design insights — all without needing physical eye-tracking hardware.
## Features
- **Saliency Heatmap** — Interactive heatmap overlay showing predicted attention intensity with adjustable opacity and colormap (Jet, Viridis, Inferno, etc.)
- **Gaze Sequence Prediction** — Numbered fixation points showing the predicted order viewers will scan the image
- **Hotspot Detection** — Top 5 attention regions ranked by intensity with bounding boxes
- **Attention Score** — Overall 0-100 concentration score measuring how focused or diffuse the predicted attention is
- **Areas of Interest (AOI)** — Draw rectangles over design elements to measure attention %, area %, and attention density
- **Rule-Based Insights** — Automatic analysis of attention concentration, focal dominance, gaze entry point, spatial balance, edge risk, and drop-off
- **AI Design Analysis** — Optional Claude Sonnet 4.6-powered insights that reference specific visual elements in your design with actionable recommendations
- **PDF Reports** — Professional downloadable reports with Montserrat typography, all visualizations, metrics, and insights (both rule-based and AI)
- **Multi-Model Support** — Choose between DeepGaze I, DeepGaze IIE (recommended), and DeepGaze III
- **Project Organization** — Group analyses into projects for easy management
- **Comparison View** — Side-by-side comparison of two analyses with metrics
## Tech Stack
| Layer | Technology |
|-------|-----------|
| **Frontend** | React 18, TypeScript, Vite, Tailwind CSS, Zustand, React Router |
| **Backend** | FastAPI, Python 3.12, SQLAlchemy (async), Pydantic v2 |
| **Database** | PostgreSQL 16 |
| **ML Models** | DeepGaze I / IIE / III via [deepgaze_pytorch](https://github.com/matthias-k/DeepGaze) |
| **AI Insights** | Anthropic Claude Sonnet 4.6 (optional) |
| **PDF Generation** | ReportLab with Montserrat font |
| **Deployment** | Docker Compose |
## Prerequisites
- Python 3.12+
- Node.js 18+
- Docker & Docker Compose (for PostgreSQL)
- Git
## Quick Start
### 1. Clone the repository
```bash
git clone git@bitbucket.org:zlalani/olivas.git
cd olivas
```
### 2. Start PostgreSQL
```bash
docker compose up -d postgres
```
This starts PostgreSQL on port **5453** with database `olivas`.
### 3. Set up the backend
```bash
cd backend
python3.12 -m venv .venv
.venv/bin/pip install --upgrade pip
.venv/bin/pip install -e ".[dev]"
.venv/bin/pip install "deepgaze-pytorch @ git+https://github.com/matthias-k/DeepGaze.git"
```
### 4. Configure environment (optional)
Create `backend/.env` for optional settings:
```env
# Required for AI Design Analysis feature (optional)
ANTHROPIC_API_KEY=sk-ant-your-key-here
# Defaults (change if needed)
DATABASE_URL=postgresql+asyncpg://olivas:olivas@localhost:5453/olivas
DEVICE=auto
CORS_ORIGINS=http://localhost:1577
```
### 5. Start the backend
```bash
cd backend
.venv/bin/uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
```
The backend will load all DeepGaze models on startup (this may take 30-60 seconds on first run as model weights are downloaded).
### 6. Set up and start the frontend
```bash
cd frontend
npm install
npm run dev
```
### 7. Open the app
Navigate to **http://localhost:1577** in your browser.
## Using the Makefile
For convenience, the project includes a Makefile:
```bash
make setup # Install all backend + frontend dependencies
make dev-backend # Start backend with hot reload
make dev-frontend # Start frontend dev server
make db-up # Start PostgreSQL container
make test # Run backend tests
make lint # Run ruff linter
make lint-fix # Auto-fix linting issues
make clean # Remove caches and virtual environments
```
## Docker Compose (Full Stack)
To run everything in Docker:
```bash
docker compose up --build
```
This starts PostgreSQL, the backend API, and the frontend. Access the app at **http://localhost:1577**.
## Project Structure
```
olivas/
├── backend/
│ ├── app/
│ │ ├── api/endpoints/ # FastAPI route handlers
│ │ ├── db/ # Database session & connection
│ │ ├── models/ # SQLAlchemy ORM models
│ │ ├── schemas/ # Pydantic request/response schemas
│ │ ├── services/
│ │ │ ├── saliency/ # DeepGaze model manager & inference
│ │ │ ├── ai_insights.py # Claude AI integration
│ │ │ ├── insights.py # Rule-based insights engine
│ │ │ ├── report_generator.py # PDF report generation
│ │ │ ├── heatmap.py # Heatmap overlay generation
│ │ │ ├── gaze_sequence.py # Gaze sequence extraction
│ │ │ ├── image_processing.py # Image resize & upscale
│ │ │ └── storage.py # File storage abstraction
│ │ ├── config.py # App settings (env vars)
│ │ └── main.py # FastAPI app entry point
│ └── pyproject.toml
├── frontend/
│ ├── src/
│ │ ├── api/ # Axios API client & endpoints
│ │ ├── components/
│ │ │ ├── analysis/ # Heatmap, gaze, hotspots, insights
│ │ │ ├── aoi/ # Area of Interest canvas & results
│ │ │ ├── common/ # Button, Card, LoadingSpinner
│ │ │ └── layout/ # Header, Sidebar, AppLayout
│ │ ├── hooks/ # React Query hooks
│ │ ├── pages/ # Dashboard, NewAnalysis, AnalysisView, Help, About
│ │ ├── stores/ # Zustand state management
│ │ └── types/ # TypeScript interfaces
│ └── package.json
├── docker-compose.yml # Production Docker setup
├── docker-compose.dev.yml # Development Docker overrides
├── Makefile # Development shortcuts
└── LICENSE # MIT License
```
## Saliency Models
OliVAS uses the [DeepGaze](https://github.com/matthias-k/DeepGaze) family of saliency prediction models:
| Model | Architecture | Best For | Reference |
|-------|-------------|----------|-----------|
| **DeepGaze IIE** (recommended) | ResNet + DenseNet ensemble | Best accuracy on benchmarks | [Linardos et al., ICCV 2021](https://arxiv.org/abs/2105.12441) |
| **DeepGaze III** | Transformer-based | Complex layouts with many elements | [Kummerer et al., J. Vision 2022](https://doi.org/10.1167/jov.22.5.7) |
| **DeepGaze I** | AlexNet features | Quick preliminary analysis | [Kummerer et al., ICLR 2015](https://arxiv.org/abs/1411.1045) |
These models are trained on thousands of real eye-tracking experiments and are among the top-performing models on the [MIT/Tubingen Saliency Benchmark](https://saliency.tuebingen.ai/).
## AI Design Analysis
When an Anthropic API key is configured, OliVAS can send the original image and heatmap overlay to **Claude Sonnet 4.6** for context-aware design analysis. The AI references specific visual elements in your design and provides actionable recommendations.
- Cost per analysis is tracked and displayed (typically $0.01-0.05 per image)
- AI insights are saved to the database and included in PDF reports
- This feature is entirely optional — rule-based insights always work without an API key
## API Endpoints
| Method | Endpoint | Description |
|--------|---------|-------------|
| `POST` | `/api/projects` | Create a new project |
| `GET` | `/api/projects` | List all projects |
| `POST` | `/api/projects/{id}/analyses` | Upload image and start analysis |
| `GET` | `/api/analyses/{id}` | Get analysis details + insights |
| `GET` | `/api/analyses/{id}/status` | Poll analysis status |
| `GET` | `/api/analyses/{id}/images/{type}` | Get analysis images |
| `POST` | `/api/analyses/{id}/ai-insights` | Generate AI insights (on-demand) |
| `GET` | `/api/analyses/{id}/report` | Download PDF report |
| `POST` | `/api/analyses/{id}/aois` | Create Areas of Interest |
| `DELETE` | `/api/analyses/{id}` | Delete an analysis |
## Academic References
- Kummerer, M., Theis, L., & Bethge, M. (2015). "Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet." *ICLR 2015*. [arXiv:1411.1045](https://arxiv.org/abs/1411.1045)
- Kummerer, M., Wallis, T.S.A., & Bethge, M. (2016). "DeepGaze II: Reading fixations from deep features trained on object recognition." [arXiv:1610.01563](https://arxiv.org/abs/1610.01563)
- Linardos, A., Kummerer, M., Press, O., & Bethge, M. (2021). "DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling." *ICCV 2021*. [arXiv:2105.12441](https://arxiv.org/abs/2105.12441)
- Kummerer, M., Bethge, M., & Wallis, T.S.A. (2022). "DeepGaze III: Modeling free-viewing human scanpaths with deep learning." *Journal of Vision*, 22(5):7. [DOI:10.1167/jov.22.5.7](https://doi.org/10.1167/jov.22.5.7)
- Itti, L., Koch, C., & Niebur, E. (1998). "A Model of Saliency-Based Visual Attention for Rapid Scene Analysis." *IEEE TPAMI*, 20(11), 1254-1259. [DOI:10.1109/34.730558](https://doi.org/10.1109/34.730558)
## License
MIT License. See [LICENSE](LICENSE) for details.
---
Built with care by **OLIVER** creative teams.

1
backend/=0.40 Normal file
View file

@ -0,0 +1 @@
(eval):1: command not found: pip

18
backend/Dockerfile Normal file
View file

@ -0,0 +1,18 @@
FROM python:3.12-slim
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
libgl1-mesa-glx libglib2.0-0 curl git && \
rm -rf /var/lib/apt/lists/*
COPY pyproject.toml .
RUN pip install --no-cache-dir -e . && \
pip install --no-cache-dir "deepgaze-pytorch @ git+https://github.com/matthias-k/DeepGaze.git"
COPY . .
RUN mkdir -p data/uploads
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

36
backend/alembic.ini Normal file
View file

@ -0,0 +1,36 @@
[alembic]
script_location = app/db/migrations
sqlalchemy.url = postgresql+asyncpg://olivas:olivas@localhost:5453/olivas
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

0
backend/app/__init__.py Normal file
View file

0
backend/app/api/__init__.py Normal file
View file

0
backend/app/api/endpoints/__init__.py Normal file
View file

452
backend/app/api/endpoints/analysis.py Normal file
View file

@ -0,0 +1,452 @@
import io
from fastapi import APIRouter, BackgroundTasks, Depends, Header, HTTPException, UploadFile, Form
from fastapi.responses import StreamingResponse
from PIL import Image
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.session import get_db
from app.dependencies import get_user_id
from app.models.analysis import Analysis
from app.models.project import Project
from app.schemas.analysis import AnalysisDetail, AnalysisStatus, AnalysisSummary
from app.services.storage import storage
router = APIRouter(tags=["analysis"])
ALLOWED_FORMATS = {"JPEG", "PNG", "TIFF", "WEBP", "BMP"}
MAX_FILE_SIZE = 50 * 1024 * 1024 # 50MB
@router.post("/projects/{project_id}/analyses", response_model=AnalysisStatus, status_code=202)
async def create_analysis(
project_id: str,
file: UploadFile,
background_tasks: BackgroundTasks,
name: str | None = Form(None),
model: str = Form("deepgaze_iie"),
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
# Verify project belongs to user
stmt = select(Project).where(Project.id == project_id, Project.user_id == user_id)
result = await db.execute(stmt)
project = result.scalar_one_or_none()
if not project:
raise HTTPException(status_code=404, detail="Project not found")
# Read and validate image
image_data = await file.read()
if len(image_data) > MAX_FILE_SIZE:
raise HTTPException(status_code=413, detail="File too large (max 50MB)")
try:
image = Image.open(io.BytesIO(image_data))
image.verify()
image = Image.open(io.BytesIO(image_data)) # re-open after verify
except Exception:
raise HTTPException(status_code=400, detail="Invalid image file")
if image.format not in ALLOWED_FORMATS:
raise HTTPException(
status_code=400,
detail=f"Unsupported format: {image.format}. Allowed: {', '.join(ALLOWED_FORMATS)}",
)
# Create analysis record
analysis = Analysis(
project_id=project_id,
user_id=user_id,
name=name or file.filename or "Untitled",
model_used=model,
status="pending",
original_filename=file.filename or "upload",
image_width=image.width,
image_height=image.height,
file_format=image.format or "PNG",
storage_path=str(storage.base_dir),
)
db.add(analysis)
await db.flush()
await db.refresh(analysis)
# Save original image
await storage.save_bytes(image_data, analysis.id, "original.png")
# Save thumbnail
thumb = image.copy()
thumb.thumbnail((400, 400))
thumb_buffer = io.BytesIO()
thumb.save(thumb_buffer, format="PNG")
await storage.save_bytes(thumb_buffer.getvalue(), analysis.id, "thumbnail.png")
analysis_id = analysis.id
# Commit now so the background thread can see the record
await db.commit()
# Queue background processing (sync function runs in threadpool)
background_tasks.add_task(run_analysis_pipeline, analysis_id, image_data, model)
return AnalysisStatus(id=analysis_id, status="pending")
def run_analysis_pipeline(analysis_id: str, image_data: bytes, model_name: str):
"""Background task: full saliency analysis pipeline. Runs sync in threadpool."""
import asyncio
import logging
import numpy as np
from app.services.saliency.model_manager import model_manager
from app.services.image_processing import prepare_for_inference, upscale_saliency
from app.services.heatmap import generate_heatmap_overlay, generate_standalone_heatmap
from app.services.gaze_sequence import extract_gaze_sequence
logger = logging.getLogger("olivas.pipeline")
# Use sync DB connection for background thread
from sqlalchemy import create_engine
from sqlalchemy.orm import Session
from app.config import settings
sync_url = settings.DATABASE_URL.replace("+asyncpg", "").replace("postgresql://", "postgresql+psycopg2://")
# Use psycopg2 if available, otherwise fallback
try:
sync_engine = create_engine(settings.DATABASE_URL.replace("+asyncpg", "+psycopg2"))
except Exception:
sync_engine = create_engine(settings.DATABASE_URL.replace("+asyncpg", ""))
try:
with Session(sync_engine) as db:
analysis = db.get(Analysis, analysis_id)
analysis.status = "processing"
db.commit()
logger.info(f"Starting analysis {analysis_id}")
image = Image.open(io.BytesIO(image_data)).convert("RGB")
# 1. Resize for inference
resized, scale = prepare_for_inference(image)
logger.info(f"Image resized: {image.size} -> {resized.size}")
# 2. Run saliency model
logger.info(f"Running {model_name} inference...")
saliency = model_manager.predict(resized, model_name)
logger.info("Inference complete")
# 3. Upscale to original dimensions
saliency_full = upscale_saliency(saliency, image.height, image.width)
# 4. Save raw saliency as .npy
np.save(str(storage.get_path(analysis_id, "saliency_raw.npy")), saliency_full)
# 5. Save saliency as grayscale PNG
saliency_uint8 = (saliency_full * 255).astype(np.uint8)
saliency_img = Image.fromarray(saliency_uint8, mode="L")
buf = io.BytesIO()
saliency_img.save(buf, format="PNG")
with open(storage.get_path(analysis_id, "saliency_gray.png"), "wb") as f:
f.write(buf.getvalue())
# 6. Generate heatmap overlay
heatmap_overlay = generate_heatmap_overlay(image, saliency_full)
buf = io.BytesIO()
heatmap_overlay.save(buf, format="PNG")
with open(storage.get_path(analysis_id, "heatmap_overlay.png"), "wb") as f:
f.write(buf.getvalue())
# 7. Generate standalone heatmap
heatmap_standalone = generate_standalone_heatmap(saliency_full)
buf = io.BytesIO()
heatmap_standalone.save(buf, format="PNG")
with open(storage.get_path(analysis_id, "heatmap_standalone.png"), "wb") as f:
f.write(buf.getvalue())
# 8. Extract gaze sequence
gaze_seq = extract_gaze_sequence(saliency_full, num_fixations=5)
# 9. Compute overall attention score
# Normalize saliency to a proper probability distribution
sal_sum = saliency_full.sum()
if sal_sum > 0:
prob_dist = saliency_full / sal_sum
prob_dist = prob_dist[prob_dist > 0] # remove zeros for log
entropy = -np.sum(prob_dist * np.log2(prob_dist))
max_entropy = np.log2(saliency_full.size)
concentration = (1 - entropy / max_entropy) * 100
overall_score = round(float(np.clip(concentration, 0, 100)), 1)
else:
overall_score = 0.0
# 10. Extract hotspots
hotspots = _extract_hotspots(saliency_full, num_hotspots=5)
# 11. Generate gaze sequence image
gaze_img = _draw_gaze_sequence(image, gaze_seq)
buf = io.BytesIO()
gaze_img.save(buf, format="PNG")
with open(storage.get_path(analysis_id, "gaze_sequence.png"), "wb") as f:
f.write(buf.getvalue())
# Update DB
analysis.status = "completed"
analysis.gaze_sequence = gaze_seq
analysis.hotspots = hotspots
analysis.overall_score = overall_score
db.commit()
logger.info(f"Analysis {analysis_id} completed (score={overall_score})")
except Exception as e:
logger.error(f"Analysis {analysis_id} failed: {e}", exc_info=True)
try:
with Session(sync_engine) as db:
analysis = db.get(Analysis, analysis_id)
if analysis:
analysis.status = "failed"
db.commit()
except Exception:
pass
def _extract_hotspots(saliency, num_hotspots=5):
import numpy as np
from scipy.ndimage import gaussian_filter
sal = saliency.copy()
h, w = sal.shape
hotspots = []
radius = int(max(h, w) * 0.08)
for i in range(num_hotspots):
smoothed = gaussian_filter(sal, sigma=max(h, w) * 0.015)
peak_idx = np.unravel_index(np.argmax(smoothed), smoothed.shape)
y, x = int(peak_idx[0]), int(peak_idx[1])
intensity = float(saliency[y, x])
# Bounding box around hotspot
x1 = max(0, x - radius)
y1 = max(0, y - radius)
x2 = min(w, x + radius)
y2 = min(h, y + radius)
hotspots.append({
"rank": i + 1,
"center_x": x,
"center_y": y,
"x": x1,
"y": y1,
"width": x2 - x1,
"height": y2 - y1,
"intensity": round(intensity, 4),
})
# Inhibition of return
yy, xx = np.ogrid[:h, :w]
mask = (xx - x) ** 2 + (yy - y) ** 2 <= radius ** 2
sal[mask] = 0.0
return hotspots
def _draw_gaze_sequence(image, gaze_seq):
from PIL import ImageDraw, ImageFont
img = image.copy()
draw = ImageDraw.Draw(img)
font = ImageFont.load_default(size=24)
colors = ["#FF4444", "#FF8800", "#FFCC00", "#44CC44", "#4488FF"]
for i, point in enumerate(gaze_seq):
x, y = point["x"], point["y"]
color = colors[i % len(colors)]
r = 25
# Draw circle
draw.ellipse([x - r, y - r, x + r, y + r], outline=color, width=3)
draw.text((x - 6, y - 12), str(point["rank"]), fill=color, font=font)
# Draw line to next point
if i < len(gaze_seq) - 1:
nx, ny = gaze_seq[i + 1]["x"], gaze_seq[i + 1]["y"]
draw.line([x, y, nx, ny], fill=color, width=2)
return img
@router.get("/analyses/ai-insights-available")
async def check_ai_insights_available():
"""Check if AI insights are available (API key configured)."""
from app.services.ai_insights import is_available
return {"available": is_available()}
@router.get("/analyses/{analysis_id}", response_model=AnalysisDetail)
async def get_analysis(
analysis_id: str,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
stmt = select(Analysis).where(Analysis.id == analysis_id, Analysis.user_id == user_id)
result = await db.execute(stmt)
analysis = result.scalar_one_or_none()
if not analysis:
raise HTTPException(status_code=404, detail="Analysis not found")
# Generate insights for completed analyses
insights = None
if analysis.status == "completed":
from app.services.insights import generate_insights
insights = generate_insights(analysis)
return AnalysisDetail(
id=analysis.id,
name=analysis.name,
model_used=analysis.model_used,
status=analysis.status,
original_filename=analysis.original_filename,
image_width=analysis.image_width,
image_height=analysis.image_height,
file_format=analysis.file_format,
overall_score=analysis.overall_score,
created_at=analysis.created_at,
gaze_sequence=analysis.gaze_sequence,
hotspots=analysis.hotspots,
insights=insights,
ai_insights=analysis.ai_insights,
ai_cost_usd=analysis.ai_cost_usd,
)
@router.get("/analyses/{analysis_id}/status", response_model=AnalysisStatus)
async def get_analysis_status(
analysis_id: str,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
stmt = select(Analysis).where(Analysis.id == analysis_id, Analysis.user_id == user_id)
result = await db.execute(stmt)
analysis = result.scalar_one_or_none()
if not analysis:
raise HTTPException(status_code=404, detail="Analysis not found")
return AnalysisStatus(id=analysis.id, status=analysis.status)
@router.get("/analyses/{analysis_id}/images/{image_type}")
async def get_analysis_image(
analysis_id: str,
image_type: str,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
stmt = select(Analysis).where(Analysis.id == analysis_id, Analysis.user_id == user_id)
result = await db.execute(stmt)
analysis = result.scalar_one_or_none()
if not analysis:
raise HTTPException(status_code=404, detail="Analysis not found")
file_map = {
"original": "original.png",
"thumbnail": "thumbnail.png",
"heatmap": "heatmap_overlay.png",
"heatmap-standalone": "heatmap_standalone.png",
"saliency-raw": "saliency_gray.png",
"gaze-sequence": "gaze_sequence.png",
}
filename = file_map.get(image_type)
if not filename:
raise HTTPException(status_code=400, detail=f"Unknown image type: {image_type}")
if not storage.exists(analysis_id, filename):
raise HTTPException(status_code=404, detail=f"Image not yet available")
data = await storage.load_bytes(analysis_id, filename)
return StreamingResponse(io.BytesIO(data), media_type="image/png")
@router.post("/analyses/{analysis_id}/ai-insights")
async def generate_ai_insights_endpoint(
analysis_id: str,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
"""Generate AI-powered insights for a completed analysis using Claude."""
from app.services.ai_insights import generate_ai_insights, is_available
if not is_available():
raise HTTPException(status_code=503, detail="AI insights not configured (missing API key)")
user_id = get_user_id(x_user_id)
stmt = select(Analysis).where(Analysis.id == analysis_id, Analysis.user_id == user_id)
result = await db.execute(stmt)
analysis = result.scalar_one_or_none()
if not analysis:
raise HTTPException(status_code=404, detail="Analysis not found")
if analysis.status != "completed":
raise HTTPException(status_code=400, detail="Analysis is not yet completed")
# Load images
try:
original_bytes = await storage.load_bytes(analysis_id, "original.png")
heatmap_bytes = await storage.load_bytes(analysis_id, "heatmap_overlay.png")
except FileNotFoundError:
raise HTTPException(status_code=404, detail="Analysis images not found")
metadata = {
"overall_score": analysis.overall_score,
"hotspots": analysis.hotspots or [],
"gaze_sequence": analysis.gaze_sequence or [],
"image_width": analysis.image_width,
"image_height": analysis.image_height,
}
try:
result = generate_ai_insights(metadata, original_bytes, heatmap_bytes)
# Save to DB
analysis.ai_insights = result["insights"]
analysis.ai_cost_usd = result["cost_usd"]
await db.flush()
# Invalidate cached PDF so next download includes AI insights
if storage.exists(analysis_id, "report.pdf"):
import os
try:
os.remove(storage.get_path(analysis_id, "report.pdf"))
except OSError:
pass
return {
"insights": result["insights"],
"cost_usd": result["cost_usd"],
"input_tokens": result["input_tokens"],
"output_tokens": result["output_tokens"],
}
except RuntimeError as e:
raise HTTPException(status_code=502, detail=str(e))
@router.delete("/analyses/{analysis_id}", status_code=204)
async def delete_analysis(
analysis_id: str,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
stmt = select(Analysis).where(Analysis.id == analysis_id, Analysis.user_id == user_id)
result = await db.execute(stmt)
analysis = result.scalar_one_or_none()
if not analysis:
raise HTTPException(status_code=404, detail="Analysis not found")
await storage.delete_analysis(analysis_id)
await db.delete(analysis)
await db.commit()

105
backend/app/api/endpoints/aoi.py Normal file
View file

@ -0,0 +1,105 @@
import numpy as np
from fastapi import APIRouter, Depends, Header, HTTPException
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.session import get_db
from app.dependencies import get_user_id
from app.models.analysis import Analysis
from app.models.aoi import AOI
from app.schemas.aoi import AOICreate, AOIResult, AOIUpdate
from app.services.aoi_analysis import compute_aoi_attention
from app.services.storage import storage
router = APIRouter(tags=["aoi"])
@router.post("/analyses/{analysis_id}/aois", response_model=list[AOIResult], status_code=201)
async def create_aois(
analysis_id: str,
body: AOICreate,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
stmt = select(Analysis).where(Analysis.id == analysis_id, Analysis.user_id == user_id)
result = await db.execute(stmt)
analysis = result.scalar_one_or_none()
if not analysis:
raise HTTPException(status_code=404, detail="Analysis not found")
if analysis.status != "completed":
raise HTTPException(status_code=400, detail="Analysis not yet completed")
# Load saliency map
npy_path = storage.get_path(analysis_id, "saliency_raw.npy")
if not npy_path.exists():
raise HTTPException(status_code=400, detail="Saliency data not available")
saliency = np.load(str(npy_path))
regions = [r.model_dump() for r in body.regions]
attention_results = compute_aoi_attention(saliency, regions)
aoi_records = []
for region, att in zip(body.regions, attention_results):
aoi = AOI(
analysis_id=analysis_id,
label=region.label,
x=region.x,
y=region.y,
width=region.width,
height=region.height,
attention_pct=att["attention_pct"],
area_pct=att["area_pct"],
attention_density=att["attention_density"],
)
db.add(aoi)
aoi_records.append(aoi)
await db.flush()
for aoi in aoi_records:
await db.refresh(aoi)
return [AOIResult.model_validate(aoi) for aoi in aoi_records]
@router.get("/analyses/{analysis_id}/aois", response_model=list[AOIResult])
async def list_aois(
analysis_id: str,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
stmt = select(Analysis).where(Analysis.id == analysis_id, Analysis.user_id == user_id)
result = await db.execute(stmt)
analysis = result.scalar_one_or_none()
if not analysis:
raise HTTPException(status_code=404, detail="Analysis not found")
aoi_stmt = select(AOI).where(AOI.analysis_id == analysis_id)
aoi_result = await db.execute(aoi_stmt)
aois = aoi_result.scalars().all()
return [AOIResult.model_validate(aoi) for aoi in aois]
@router.delete("/analyses/{analysis_id}/aois/{aoi_id}", status_code=204)
async def delete_aoi(
analysis_id: str,
aoi_id: str,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
# Verify analysis ownership
stmt = select(Analysis).where(Analysis.id == analysis_id, Analysis.user_id == user_id)
result = await db.execute(stmt)
if not result.scalar_one_or_none():
raise HTTPException(status_code=404, detail="Analysis not found")
aoi_stmt = select(AOI).where(AOI.id == aoi_id, AOI.analysis_id == analysis_id)
aoi_result = await db.execute(aoi_stmt)
aoi = aoi_result.scalar_one_or_none()
if not aoi:
raise HTTPException(status_code=404, detail="AOI not found")
await db.delete(aoi)

145
backend/app/api/endpoints/comparison.py Normal file
View file

@ -0,0 +1,145 @@
from fastapi import APIRouter, Depends, Header, HTTPException
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.session import get_db
from app.dependencies import get_user_id
from app.models.analysis import Analysis
from app.models.comparison import Comparison
from app.models.project import Project
router = APIRouter(tags=["comparison"])
class ComparisonCreate:
pass
from pydantic import BaseModel
class ComparisonCreateBody(BaseModel):
name: str
analysis_ids: list[str]
class ComparisonResponse(BaseModel):
id: str
name: str
analyses: list[dict]
comparison_data: dict | None = None
model_config = {"from_attributes": True}
@router.post(
"/projects/{project_id}/comparisons", response_model=ComparisonResponse, status_code=201
)
async def create_comparison(
project_id: str,
body: ComparisonCreateBody,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
# Verify project
stmt = select(Project).where(Project.id == project_id, Project.user_id == user_id)
result = await db.execute(stmt)
if not result.scalar_one_or_none():
raise HTTPException(status_code=404, detail="Project not found")
if len(body.analysis_ids) < 2:
raise HTTPException(status_code=400, detail="Need at least 2 analyses to compare")
# Fetch analyses
analyses_stmt = select(Analysis).where(
Analysis.id.in_(body.analysis_ids),
Analysis.user_id == user_id,
Analysis.status == "completed",
)
analyses_result = await db.execute(analyses_stmt)
analyses = analyses_result.scalars().all()
if len(analyses) != len(body.analysis_ids):
raise HTTPException(
status_code=400, detail="Some analyses not found or not completed"
)
# Build comparison data
analyses_data = []
winner_id = None
max_score = -1
for a in analyses:
data = {
"analysis_id": a.id,
"name": a.name,
"overall_score": a.overall_score,
"top_fixation": a.gaze_sequence[0] if a.gaze_sequence else None,
}
analyses_data.append(data)
if a.overall_score and a.overall_score > max_score:
max_score = a.overall_score
winner_id = a.id
comparison_data = {
"winner": winner_id,
"score_delta": round(max_score - min(a.overall_score or 0 for a in analyses), 1),
}
comparison = Comparison(
project_id=project_id,
user_id=user_id,
name=body.name,
analysis_ids=body.analysis_ids,
comparison_data=comparison_data,
)
db.add(comparison)
await db.flush()
await db.refresh(comparison)
return ComparisonResponse(
id=comparison.id,
name=comparison.name,
analyses=analyses_data,
comparison_data=comparison_data,
)
@router.get("/comparisons/{comparison_id}", response_model=ComparisonResponse)
async def get_comparison(
comparison_id: str,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
stmt = select(Comparison).where(
Comparison.id == comparison_id, Comparison.user_id == user_id
)
result = await db.execute(stmt)
comparison = result.scalar_one_or_none()
if not comparison:
raise HTTPException(status_code=404, detail="Comparison not found")
# Fetch the analyses for display
analyses_stmt = select(Analysis).where(Analysis.id.in_(comparison.analysis_ids))
analyses_result = await db.execute(analyses_stmt)
analyses = analyses_result.scalars().all()
analyses_data = [
{
"analysis_id": a.id,
"name": a.name,
"overall_score": a.overall_score,
"top_fixation": a.gaze_sequence[0] if a.gaze_sequence else None,
}
for a in analyses
]
return ComparisonResponse(
id=comparison.id,
name=comparison.name,
analyses=analyses_data,
comparison_data=comparison.comparison_data,
)

14
backend/app/api/endpoints/health.py Normal file
View file

@ -0,0 +1,14 @@
from fastapi import APIRouter
router = APIRouter()
@router.get("/health")
async def health_check():
from app.services.saliency.model_manager import model_manager
loaded = list(model_manager.models.keys()) if model_manager.models else []
return {
"status": "ok",
"models_loaded": loaded,
}

141
backend/app/api/endpoints/projects.py Normal file
View file

@ -0,0 +1,141 @@
from fastapi import APIRouter, Depends, Header, HTTPException
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.db.session import get_db
from app.dependencies import get_user_id
from app.models.analysis import Analysis
from app.models.project import Project
from app.schemas.project import ProjectCreate, ProjectDetail, ProjectSummary, ProjectUpdate
router = APIRouter(prefix="/projects", tags=["projects"])
@router.post("", response_model=ProjectSummary, status_code=201)
async def create_project(
body: ProjectCreate,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
project = Project(user_id=user_id, name=body.name, description=body.description)
db.add(project)
await db.flush()
await db.refresh(project)
return ProjectSummary(
id=project.id,
name=project.name,
description=project.description,
analysis_count=0,
created_at=project.created_at,
updated_at=project.updated_at,
)
@router.get("", response_model=list[ProjectSummary])
async def list_projects(
page: int = 1,
per_page: int = 20,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
offset = (page - 1) * per_page
stmt = (
select(
Project,
func.count(Analysis.id).label("analysis_count"),
)
.outerjoin(Analysis)
.where(Project.user_id == user_id)
.group_by(Project.id)
.order_by(Project.updated_at.desc())
.offset(offset)
.limit(per_page)
)
result = await db.execute(stmt)
rows = result.all()
return [
ProjectSummary(
id=p.id,
name=p.name,
description=p.description,
analysis_count=count,
created_at=p.created_at,
updated_at=p.updated_at,
)
for p, count in rows
]
@router.get("/{project_id}", response_model=ProjectDetail)
async def get_project(
project_id: str,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
stmt = (
select(Project)
.options(selectinload(Project.analyses))
.where(Project.id == project_id, Project.user_id == user_id)
)
result = await db.execute(stmt)
project = result.scalar_one_or_none()
if not project:
raise HTTPException(status_code=404, detail="Project not found")
return project
@router.put("/{project_id}", response_model=ProjectSummary)
async def update_project(
project_id: str,
body: ProjectUpdate,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
stmt = select(Project).where(Project.id == project_id, Project.user_id == user_id)
result = await db.execute(stmt)
project = result.scalar_one_or_none()
if not project:
raise HTTPException(status_code=404, detail="Project not found")
if body.name is not None:
project.name = body.name
if body.description is not None:
project.description = body.description
await db.flush()
await db.refresh(project)
count_stmt = select(func.count(Analysis.id)).where(Analysis.project_id == project_id)
count_result = await db.execute(count_stmt)
analysis_count = count_result.scalar() or 0
return ProjectSummary(
id=project.id,
name=project.name,
description=project.description,
analysis_count=analysis_count,
created_at=project.created_at,
updated_at=project.updated_at,
)
@router.delete("/{project_id}", status_code=204)
async def delete_project(
project_id: str,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
stmt = select(Project).where(Project.id == project_id, Project.user_id == user_id)
result = await db.execute(stmt)
project = result.scalar_one_or_none()
if not project:
raise HTTPException(status_code=404, detail="Project not found")
await db.delete(project)

89
backend/app/api/endpoints/reports.py Normal file
View file

@ -0,0 +1,89 @@
import io
import logging
from fastapi import APIRouter, Depends, Header, HTTPException
from fastapi.responses import StreamingResponse
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.db.session import get_db
from app.dependencies import get_user_id
from app.models.analysis import Analysis
from app.services.storage import storage
router = APIRouter(tags=["reports"])
logger = logging.getLogger("olivas.reports")
@router.get("/analyses/{analysis_id}/report")
async def download_report(
analysis_id: str,
db: AsyncSession = Depends(get_db),
x_user_id: str | None = Header(None),
):
user_id = get_user_id(x_user_id)
stmt = (
select(Analysis)
.options(selectinload(Analysis.aois))
.where(Analysis.id == analysis_id, Analysis.user_id == user_id)
)
result = await db.execute(stmt)
analysis = result.scalar_one_or_none()
if not analysis:
raise HTTPException(status_code=404, detail="Analysis not found")
if analysis.status != "completed":
raise HTTPException(status_code=400, detail="Analysis not yet completed")
# Check if PDF already cached
if storage.exists(analysis_id, "report.pdf"):
data = await storage.load_bytes(analysis_id, "report.pdf")
else:
from app.services.report_generator import generate_report
# Load images for the report
try:
original_data = await storage.load_bytes(analysis_id, "original.png")
except FileNotFoundError:
raise HTTPException(status_code=404, detail="Original image not found")
try:
heatmap_data = await storage.load_bytes(analysis_id, "heatmap_overlay.png")
except FileNotFoundError:
raise HTTPException(status_code=404, detail="Heatmap image not yet generated")
try:
gaze_data = await storage.load_bytes(analysis_id, "gaze_sequence.png")
except FileNotFoundError:
raise HTTPException(status_code=404, detail="Gaze sequence image not yet generated")
# Generate rule-based insights
from app.services.insights import generate_insights
rule_insights = generate_insights(analysis) if analysis.status == "completed" else []
try:
data = generate_report(
analysis=analysis,
original_image=original_data,
heatmap_image=heatmap_data,
gaze_image=gaze_data,
aois=list(analysis.aois),
rule_insights=rule_insights,
ai_insights=analysis.ai_insights,
ai_cost_usd=analysis.ai_cost_usd,
)
except Exception as e:
logger.error(f"Report generation failed for {analysis_id}: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"Report generation failed: {str(e)}")
await storage.save_bytes(data, analysis_id, "report.pdf")
safe_name = analysis.name.replace(" ", "_").replace("/", "_")
return StreamingResponse(
io.BytesIO(data),
media_type="application/pdf",
headers={
"Content-Disposition": f'attachment; filename="olivas-report-{safe_name}.pdf"'
},
)

12
backend/app/api/router.py Normal file
View file

@ -0,0 +1,12 @@
from fastapi import APIRouter
from app.api.endpoints import analysis, aoi, comparison, health, projects, reports
api_router = APIRouter(prefix="/api")
api_router.include_router(health.router)
api_router.include_router(projects.router)
api_router.include_router(analysis.router)
api_router.include_router(aoi.router)
api_router.include_router(comparison.router)
api_router.include_router(reports.router)

30
backend/app/config.py Normal file
View file

@ -0,0 +1,30 @@
from pydantic_settings import BaseSettings
class Settings(BaseSettings):
DATABASE_URL: str = "postgresql+asyncpg://olivas:olivas@localhost:5453/olivas"
UPLOAD_DIR: str = "./data/uploads"
DEVICE: str = "auto" # auto | cpu | cuda
ANTHROPIC_API_KEY: str = ""
CORS_ORIGINS: str = "http://localhost:1577"
BACKEND_HOST: str = "0.0.0.0"
BACKEND_PORT: int = 8000
@property
def device(self) -> str:
if self.DEVICE == "auto":
try:
import torch
return "cuda" if torch.cuda.is_available() else "cpu"
except ImportError:
return "cpu"
return self.DEVICE
@property
def cors_origins_list(self) -> list[str]:
return [o.strip() for o in self.CORS_ORIGINS.split(",")]
model_config = {"env_file": ".env", "env_file_encoding": "utf-8"}
settings = Settings()

0
backend/app/core/__init__.py Normal file
View file

18
backend/app/core/exceptions.py Normal file
View file

@ -0,0 +1,18 @@
class OliVASError(Exception):
pass
class AnalysisNotFoundError(OliVASError):
pass
class ProjectNotFoundError(OliVASError):
pass
class ModelNotLoadedError(OliVASError):
pass
class ImageProcessingError(OliVASError):
pass

0
backend/app/db/__init__.py Normal file
View file

57
backend/app/db/migrations/env.py Normal file
View file

@ -0,0 +1,57 @@
import asyncio
from logging.config import fileConfig
from alembic import context
from sqlalchemy import pool
from sqlalchemy.ext.asyncio import async_engine_from_config
from app.models.base import Base
from app.models.project import Project # noqa: F401
from app.models.analysis import Analysis # noqa: F401
from app.models.aoi import AOI # noqa: F401
from app.models.comparison import Comparison # noqa: F401
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
target_metadata = Base.metadata
def run_migrations_offline() -> None:
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def do_run_migrations(connection):
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
async def run_async_migrations() -> None:
connectable = async_engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
async with connectable.connect() as connection:
await connection.run_sync(do_run_migrations)
await connectable.dispose()
def run_migrations_online() -> None:
asyncio.run(run_async_migrations())
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()

26
backend/app/db/migrations/script.py.mako Normal file
View file

@ -0,0 +1,26 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
# revision identifiers, used by Alembic.
revision: str = ${repr(up_revision)}
down_revision: Union[str, None] = ${repr(down_revision)}
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
def upgrade() -> None:
${upgrades if upgrades else "pass"}
def downgrade() -> None:
${downgrades if downgrades else "pass"}

97
backend/app/db/migrations/versions/2f6f70f606a1_initial_schema.py Normal file
View file

@ -0,0 +1,97 @@
"""initial schema
Revision ID: 2f6f70f606a1
Revises:
Create Date: 2026-02-23 14:18:44.382856
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '2f6f70f606a1'
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('projects',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('user_id', sa.String(length=36), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('created_at', sa.DateTime(), server_default=sa.text('now()'), nullable=False),
sa.Column('updated_at', sa.DateTime(), server_default=sa.text('now()'), nullable=False),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_projects_user_id'), 'projects', ['user_id'], unique=False)
op.create_table('analyses',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('project_id', sa.String(length=36), nullable=False),
sa.Column('user_id', sa.String(length=36), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('model_used', sa.String(length=50), nullable=False),
sa.Column('status', sa.String(length=20), nullable=False),
sa.Column('original_filename', sa.String(length=255), nullable=False),
sa.Column('image_width', sa.Integer(), nullable=False),
sa.Column('image_height', sa.Integer(), nullable=False),
sa.Column('file_format', sa.String(length=10), nullable=False),
sa.Column('storage_path', sa.String(length=512), nullable=False),
sa.Column('gaze_sequence', sa.JSON(), nullable=True),
sa.Column('hotspots', sa.JSON(), nullable=True),
sa.Column('overall_score', sa.Float(), nullable=True),
sa.Column('created_at', sa.DateTime(), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['project_id'], ['projects.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_analyses_project_id'), 'analyses', ['project_id'], unique=False)
op.create_index(op.f('ix_analyses_user_id'), 'analyses', ['user_id'], unique=False)
op.create_table('comparisons',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('project_id', sa.String(length=36), nullable=False),
sa.Column('user_id', sa.String(length=36), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('analysis_ids', sa.JSON(), nullable=False),
sa.Column('comparison_data', sa.JSON(), nullable=True),
sa.Column('created_at', sa.DateTime(), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['project_id'], ['projects.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_comparisons_project_id'), 'comparisons', ['project_id'], unique=False)
op.create_index(op.f('ix_comparisons_user_id'), 'comparisons', ['user_id'], unique=False)
op.create_table('aois',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('analysis_id', sa.String(length=36), nullable=False),
sa.Column('label', sa.String(length=100), nullable=False),
sa.Column('x', sa.Integer(), nullable=False),
sa.Column('y', sa.Integer(), nullable=False),
sa.Column('width', sa.Integer(), nullable=False),
sa.Column('height', sa.Integer(), nullable=False),
sa.Column('attention_pct', sa.Float(), nullable=True),
sa.Column('area_pct', sa.Float(), nullable=True),
sa.Column('attention_density', sa.Float(), nullable=True),
sa.ForeignKeyConstraint(['analysis_id'], ['analyses.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_aois_analysis_id'), 'aois', ['analysis_id'], unique=False)
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.drop_index(op.f('ix_aois_analysis_id'), table_name='aois')
op.drop_table('aois')
op.drop_index(op.f('ix_comparisons_user_id'), table_name='comparisons')
op.drop_index(op.f('ix_comparisons_project_id'), table_name='comparisons')
op.drop_table('comparisons')
op.drop_index(op.f('ix_analyses_user_id'), table_name='analyses')
op.drop_index(op.f('ix_analyses_project_id'), table_name='analyses')
op.drop_table('analyses')
op.drop_index(op.f('ix_projects_user_id'), table_name='projects')
op.drop_table('projects')
# ### end Alembic commands ###

16
backend/app/db/session.py Normal file
View file

@ -0,0 +1,16 @@
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from app.config import settings
engine = create_async_engine(settings.DATABASE_URL, echo=False)
async_session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
async def get_db() -> AsyncSession:
async with async_session_factory() as session:
try:
yield session
await session.commit()
except Exception:
await session.rollback()
raise

12
backend/app/dependencies.py Normal file
View file

@ -0,0 +1,12 @@
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.session import get_db
# User ID header — placeholder for SSO integration.
# When SSO is added, this will extract user_id from the JWT/session token.
USER_ID_HEADER = "X-User-Id"
DEFAULT_USER_ID = "default"
def get_user_id(x_user_id: str | None = None) -> str:
return x_user_id or DEFAULT_USER_ID

47
backend/app/main.py Normal file
View file

@ -0,0 +1,47 @@
import logging
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from app.api.router import api_router
from app.config import settings
logger = logging.getLogger("olivas")
@asynccontextmanager
async def lifespan(app: FastAPI):
# Startup: load ML models
logger.info(f"Starting OliVAS backend (device={settings.device})")
from app.services.saliency.model_manager import model_manager
try:
model_manager.load_models(device=settings.device)
logger.info(f"Models loaded: {list(model_manager.models.keys())}")
except Exception as e:
logger.warning(f"Failed to load ML models: {e}. Analysis will fail until models load.")
yield
# Shutdown
model_manager.cleanup()
logger.info("OliVAS backend shut down")
app = FastAPI(
title="OliVAS",
description="Open-Source Visual Attention Software by OLIVER",
version="0.1.0",
lifespan=lifespan,
)
app.add_middleware(
CORSMiddleware,
allow_origins=settings.cors_origins_list,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
app.include_router(api_router)

0
backend/app/models/__init__.py Normal file
View file

39
backend/app/models/analysis.py Normal file
View file

@ -0,0 +1,39 @@
import uuid
from datetime import datetime
from sqlalchemy import Float, ForeignKey, Integer, JSON, String, func
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.models.base import Base
class Analysis(Base):
__tablename__ = "analyses"
id: Mapped[str] = mapped_column(
String(36), primary_key=True, default=lambda: str(uuid.uuid4())
)
project_id: Mapped[str] = mapped_column(ForeignKey("projects.id"), index=True)
user_id: Mapped[str] = mapped_column(String(36), index=True)
name: Mapped[str] = mapped_column(String(255))
model_used: Mapped[str] = mapped_column(String(50), default="deepgaze_iie")
status: Mapped[str] = mapped_column(String(20), default="pending")
original_filename: Mapped[str] = mapped_column(String(255))
image_width: Mapped[int] = mapped_column(Integer)
image_height: Mapped[int] = mapped_column(Integer)
file_format: Mapped[str] = mapped_column(String(10))
storage_path: Mapped[str] = mapped_column(String(512))
gaze_sequence: Mapped[dict | None] = mapped_column(JSON, nullable=True)
hotspots: Mapped[dict | None] = mapped_column(JSON, nullable=True)
overall_score: Mapped[float | None] = mapped_column(Float, nullable=True)
ai_insights: Mapped[dict | None] = mapped_column(JSON, nullable=True)
ai_cost_usd: Mapped[float | None] = mapped_column(Float, nullable=True)
created_at: Mapped[datetime] = mapped_column(server_default=func.now())
project: Mapped["Project"] = relationship(back_populates="analyses") # noqa: F821
aois: Mapped[list["AOI"]] = relationship( # noqa: F821
back_populates="analysis", cascade="all, delete-orphan"
)

26
backend/app/models/aoi.py Normal file
View file

@ -0,0 +1,26 @@
import uuid
from sqlalchemy import Float, ForeignKey, Integer, String
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.models.base import Base
class AOI(Base):
__tablename__ = "aois"
id: Mapped[str] = mapped_column(
String(36), primary_key=True, default=lambda: str(uuid.uuid4())
)
analysis_id: Mapped[str] = mapped_column(ForeignKey("analyses.id"), index=True)
label: Mapped[str] = mapped_column(String(100))
x: Mapped[int] = mapped_column(Integer)
y: Mapped[int] = mapped_column(Integer)
width: Mapped[int] = mapped_column(Integer)
height: Mapped[int] = mapped_column(Integer)
attention_pct: Mapped[float | None] = mapped_column(Float, nullable=True)
area_pct: Mapped[float | None] = mapped_column(Float, nullable=True)
attention_density: Mapped[float | None] = mapped_column(Float, nullable=True)
analysis: Mapped["Analysis"] = relationship(back_populates="aois") # noqa: F821

5
backend/app/models/base.py Normal file
View file

@ -0,0 +1,5 @@
from sqlalchemy.orm import DeclarativeBase
class Base(DeclarativeBase):
pass

23
backend/app/models/comparison.py Normal file
View file

@ -0,0 +1,23 @@
import uuid
from datetime import datetime
from sqlalchemy import ForeignKey, JSON, String, func
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.models.base import Base
class Comparison(Base):
__tablename__ = "comparisons"
id: Mapped[str] = mapped_column(
String(36), primary_key=True, default=lambda: str(uuid.uuid4())
)
project_id: Mapped[str] = mapped_column(ForeignKey("projects.id"), index=True)
user_id: Mapped[str] = mapped_column(String(36), index=True)
name: Mapped[str] = mapped_column(String(255))
analysis_ids: Mapped[dict] = mapped_column(JSON)
comparison_data: Mapped[dict | None] = mapped_column(JSON, nullable=True)
created_at: Mapped[datetime] = mapped_column(server_default=func.now())
project: Mapped["Project"] = relationship(back_populates="comparisons") # noqa: F821

27
backend/app/models/project.py Normal file
View file

@ -0,0 +1,27 @@
import uuid
from datetime import datetime
from sqlalchemy import String, Text, func
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.models.base import Base
class Project(Base):
__tablename__ = "projects"
id: Mapped[str] = mapped_column(
String(36), primary_key=True, default=lambda: str(uuid.uuid4())
)
user_id: Mapped[str] = mapped_column(String(36), index=True)
name: Mapped[str] = mapped_column(String(255))
description: Mapped[str | None] = mapped_column(Text, nullable=True)
created_at: Mapped[datetime] = mapped_column(server_default=func.now())
updated_at: Mapped[datetime] = mapped_column(server_default=func.now(), onupdate=func.now())
analyses: Mapped[list["Analysis"]] = relationship( # noqa: F821
back_populates="project", cascade="all, delete-orphan"
)
comparisons: Mapped[list["Comparison"]] = relationship( # noqa: F821
back_populates="project", cascade="all, delete-orphan"
)

0
backend/app/schemas/__init__.py Normal file
View file

52
backend/app/schemas/analysis.py Normal file
View file

@ -0,0 +1,52 @@
from datetime import datetime
from pydantic import BaseModel
class AnalysisSummary(BaseModel):
id: str
name: str
model_used: str
status: str
original_filename: str
image_width: int
image_height: int
overall_score: float | None = None
created_at: datetime
model_config = {"from_attributes": True}
class GazePoint(BaseModel):
rank: int
x: int
y: int
x_pct: float
y_pct: float
probability: float
class Insight(BaseModel):
type: str # "info" | "success" | "warning"
title: str
description: str
class AnalysisDetail(AnalysisSummary):
file_format: str
gaze_sequence: list[GazePoint] | None = None
hotspots: list[dict] | None = None
insights: list[Insight] | None = None
ai_insights: list[Insight] | None = None
ai_cost_usd: float | None = None
aoi_count: int = 0
class AnalysisCreate(BaseModel):
name: str | None = None
model: str = "deepgaze_iie"
class AnalysisStatus(BaseModel):
id: str
status: str

35
backend/app/schemas/aoi.py Normal file
View file

@ -0,0 +1,35 @@
from pydantic import BaseModel
class AOIRegion(BaseModel):
label: str
x: int
y: int
width: int
height: int
class AOICreate(BaseModel):
regions: list[AOIRegion]
class AOIResult(BaseModel):
id: str
label: str
x: int
y: int
width: int
height: int
attention_pct: float | None = None
area_pct: float | None = None
attention_density: float | None = None
model_config = {"from_attributes": True}
class AOIUpdate(BaseModel):
label: str | None = None
x: int | None = None
y: int | None = None
width: int | None = None
height: int | None = None

35
backend/app/schemas/project.py Normal file
View file

@ -0,0 +1,35 @@
from datetime import datetime
from pydantic import BaseModel
class ProjectCreate(BaseModel):
name: str
description: str | None = None
class ProjectUpdate(BaseModel):
name: str | None = None
description: str | None = None
class ProjectSummary(BaseModel):
id: str
name: str
description: str | None
analysis_count: int = 0
created_at: datetime
updated_at: datetime
model_config = {"from_attributes": True}
class ProjectDetail(ProjectSummary):
analyses: list["AnalysisSummary"] = []
model_config = {"from_attributes": True}
from app.schemas.analysis import AnalysisSummary # noqa: E402
ProjectDetail.model_rebuild()

7
backend/app/schemas/report.py Normal file
View file

@ -0,0 +1,7 @@
from pydantic import BaseModel
class ReportRequest(BaseModel):
include_aois: bool = True
include_gaze_sequence: bool = True
include_hotspots: bool = True

0
backend/app/services/__init__.py Normal file
View file

158
backend/app/services/ai_insights.py Normal file
View file

@ -0,0 +1,158 @@
"""Claude-powered AI insights for saliency analysis."""
import base64
import json
import logging
import anthropic
from app.config import settings
logger = logging.getLogger("olivas.ai_insights")
# Sonnet 4.6 pricing per million tokens
INPUT_COST_PER_M = 3.00
OUTPUT_COST_PER_M = 15.00
def is_available() -> bool:
"""Check if AI insights are available (API key configured)."""
return bool(settings.ANTHROPIC_API_KEY)
def generate_ai_insights(
analysis_metadata: dict,
original_image_bytes: bytes,
heatmap_image_bytes: bytes,
) -> dict:
"""
Send the original image, heatmap overlay, and analysis metrics to Claude
and return actionable design insights with cost tracking.
Returns:
dict with keys: insights (list[dict]), cost_usd (float), input_tokens (int), output_tokens (int)
"""
if not is_available():
raise RuntimeError("ANTHROPIC_API_KEY is not configured")
client = anthropic.Anthropic(api_key=settings.ANTHROPIC_API_KEY)
original_b64 = base64.standard_b64encode(original_image_bytes).decode("utf-8")
heatmap_b64 = base64.standard_b64encode(heatmap_image_bytes).decode("utf-8")
score = analysis_metadata.get("overall_score", 0)
hotspots = analysis_metadata.get("hotspots", [])
gaze_sequence = analysis_metadata.get("gaze_sequence", [])
width = analysis_metadata.get("image_width", 0)
height = analysis_metadata.get("image_height", 0)
metrics_text = f"""Analysis Metrics:
- Image dimensions: {width} x {height} pixels
- Overall Attention Score: {score}/100 (higher = more concentrated attention)
- Number of hotspots detected: {len(hotspots)}
- Top hotspot location: ({hotspots[0]['center_x']}, {hotspots[0]['center_y']}) with intensity {hotspots[0]['intensity']:.4f}
- First gaze fixation point: ({gaze_sequence[0]['x']}, {gaze_sequence[0]['y']}) — this is where viewers look first
""" if hotspots and gaze_sequence else f"""Analysis Metrics:
- Image dimensions: {width} x {height} pixels
- Overall Attention Score: {score}/100
"""
prompt = f"""You are a visual attention and design expert analyzing an image using saliency prediction data. You have been given:
1. The original image
2. A heatmap overlay showing predicted visual attention (warm colors = high attention, cool colors = low attention)
3. Numerical metrics from the analysis
{metrics_text}
Based on the original image, the heatmap overlay, and the metrics, provide exactly 4 actionable design insights. Each insight should be specific to THIS image — reference actual visual elements you can see (logos, text, products, faces, backgrounds, etc.).
For each insight, provide:
- "type": one of "success" (something working well), "info" (neutral observation or suggestion), or "warning" (potential issue)
- "title": a short heading (5-10 words)
- "description": 1-2 sentences of specific, actionable advice
Respond with ONLY a JSON array of 4 insight objects. No other text, no markdown formatting, just the JSON array.
Example format:
[
{{"type": "success", "title": "Strong focal point on product", "description": "The product image captures the highest predicted attention, which aligns well with the likely design goal. The bright contrast against the background naturally draws the eye."}},
{{"type": "warning", "title": "CTA button may be overlooked", "description": "The call-to-action button in the lower right receives minimal predicted attention. Consider increasing its size, contrast, or moving it closer to the primary focal area."}}
]"""
try:
message = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=1024,
messages=[
{
"role": "user",
"content": [
{
"type": "image",
"source": {
"type": "base64",
"media_type": "image/png",
"data": original_b64,
},
},
{
"type": "image",
"source": {
"type": "base64",
"media_type": "image/png",
"data": heatmap_b64,
},
},
{
"type": "text",
"text": prompt,
},
],
}
],
)
# Extract token usage and compute cost
input_tokens = message.usage.input_tokens
output_tokens = message.usage.output_tokens
cost_usd = round(
(input_tokens / 1_000_000) * INPUT_COST_PER_M
+ (output_tokens / 1_000_000) * OUTPUT_COST_PER_M,
6,
)
response_text = message.content[0].text.strip()
# Handle potential markdown code block wrapping
if response_text.startswith("```"):
response_text = response_text.split("\n", 1)[1]
if response_text.endswith("```"):
response_text = response_text[:-3].strip()
insights = json.loads(response_text)
# Validate structure
validated = []
for item in insights[:5]: # cap at 5
if isinstance(item, dict) and "type" in item and "title" in item and "description" in item:
if item["type"] not in ("success", "info", "warning"):
item["type"] = "info"
validated.append({
"type": item["type"],
"title": str(item["title"]),
"description": str(item["description"]),
})
return {
"insights": validated,
"cost_usd": cost_usd,
"input_tokens": input_tokens,
"output_tokens": output_tokens,
}
except anthropic.APIError as e:
logger.error(f"Anthropic API error: {e}")
raise RuntimeError(f"AI analysis failed: {e.message}")
except json.JSONDecodeError as e:
logger.error(f"Failed to parse AI response: {e}")
raise RuntimeError("AI returned an unexpected response format")

47
backend/app/services/aoi_analysis.py Normal file
View file

@ -0,0 +1,47 @@
import numpy as np
def compute_aoi_attention(
saliency: np.ndarray,
regions: list[dict],
) -> list[dict]:
"""
Compute attention metrics for user-defined regions.
Returns list of dicts with attention_pct, area_pct, attention_density.
"""
h, w = saliency.shape
total_saliency = saliency.sum()
total_area = h * w
if total_saliency == 0:
return [
{"attention_pct": 0.0, "area_pct": 0.0, "attention_density": 0.0}
for _ in regions
]
results = []
for region in regions:
rx, ry = region["x"], region["y"]
rw, rh = region["width"], region["height"]
# Clamp to image bounds
x1 = max(0, rx)
y1 = max(0, ry)
x2 = min(w, rx + rw)
y2 = min(h, ry + rh)
region_saliency = saliency[y1:y2, x1:x2].sum()
region_area = (x2 - x1) * (y2 - y1)
attention_pct = (region_saliency / total_saliency) * 100
area_pct = (region_area / total_area) * 100
density = attention_pct / max(area_pct, 0.01)
results.append({
"attention_pct": round(float(attention_pct), 2),
"area_pct": round(float(area_pct), 2),
"attention_density": round(float(density), 2),
})
return results

BIN
backend/app/services/fonts/Montserrat-Bold.ttf Normal file
View file

Binary file not shown.

BIN
backend/app/services/fonts/Montserrat-Regular.ttf Normal file
View file

Binary file not shown.

BIN
backend/app/services/fonts/Montserrat-SemiBold.ttf Normal file
View file

Binary file not shown.

42
backend/app/services/gaze_sequence.py Normal file
View file

@ -0,0 +1,42 @@
import numpy as np
from scipy.ndimage import gaussian_filter
def extract_gaze_sequence(
saliency: np.ndarray,
num_fixations: int = 5,
inhibition_radius_fraction: float = 0.1,
) -> list[dict]:
"""
Extract predicted fixation sequence using iterative peak finding
with inhibition-of-return.
"""
sal = saliency.copy().astype(np.float64)
h, w = sal.shape
inhibition_radius = int(max(h, w) * inhibition_radius_fraction)
fixations = []
for rank in range(1, num_fixations + 1):
smoothed = gaussian_filter(sal, sigma=max(h, w) * 0.01)
if smoothed.max() < 1e-10:
break
peak_idx = np.unravel_index(np.argmax(smoothed), smoothed.shape)
y, x = int(peak_idx[0]), int(peak_idx[1])
prob = float(saliency[y, x])
fixations.append({
"rank": rank,
"x": x,
"y": y,
"x_pct": round(x / w * 100, 1),
"y_pct": round(y / h * 100, 1),
"probability": round(prob, 4),
})
# Inhibition of return
yy, xx = np.ogrid[:h, :w]
mask = (xx - x) ** 2 + (yy - y) ** 2 <= inhibition_radius ** 2
sal[mask] = 0.0
return fixations

26
backend/app/services/heatmap.py Normal file
View file

@ -0,0 +1,26 @@
import matplotlib
matplotlib.use("Agg")
import matplotlib.cm as cm
import numpy as np
from PIL import Image
def generate_heatmap_overlay(
original: Image.Image,
saliency: np.ndarray,
colormap: str = "jet",
alpha: float = 0.5,
) -> Image.Image:
"""Generate heatmap blended over original image."""
cmap = matplotlib.colormaps.get_cmap(colormap)
heatmap_rgba = cmap(saliency)
heatmap_rgb = (heatmap_rgba[:, :, :3] * 255).astype(np.uint8)
heatmap_img = Image.fromarray(heatmap_rgb).resize(original.size, Image.LANCZOS)
return Image.blend(original.convert("RGB"), heatmap_img, alpha)
def generate_standalone_heatmap(saliency: np.ndarray, colormap: str = "jet") -> Image.Image:
"""Generate a pure heatmap image."""
cmap = matplotlib.colormaps.get_cmap(colormap)
heatmap_rgba = cmap(saliency)
return Image.fromarray((heatmap_rgba[:, :, :3] * 255).astype(np.uint8))

24
backend/app/services/image_processing.py Normal file
View file

@ -0,0 +1,24 @@
import numpy as np
from PIL import Image
from scipy.ndimage import zoom
MAX_INFERENCE_SIZE = 1024
def prepare_for_inference(image: Image.Image) -> tuple[Image.Image, float]:
"""Resize for model input. Returns (resized_image, scale_factor)."""
w, h = image.size
scale = MAX_INFERENCE_SIZE / max(w, h)
if scale < 1.0:
new_size = (int(w * scale), int(h * scale))
return image.resize(new_size, Image.LANCZOS), scale
return image, 1.0
def upscale_saliency(saliency: np.ndarray, target_h: int, target_w: int) -> np.ndarray:
"""Upscale saliency map to original image dimensions."""
if saliency.shape == (target_h, target_w):
return saliency
h_scale = target_h / saliency.shape[0]
w_scale = target_w / saliency.shape[1]
return zoom(saliency, (h_scale, w_scale), order=1)

200
backend/app/services/insights.py Normal file
View file

@ -0,0 +1,200 @@
"""Rule-based insights engine for visual attention analysis.
Generates actionable text insights from analysis metrics —
no AI needed, just conditional logic on the computed data.
"""
def generate_insights(analysis) -> list[dict]:
"""Generate insights from a completed analysis.
Args:
analysis: Analysis ORM object with overall_score, hotspots, gaze_sequence,
image_width, image_height.
Returns:
List of dicts with keys: type (info|success|warning), title, description.
"""
insights = []
score = analysis.overall_score
hotspots = analysis.hotspots or []
gaze_seq = analysis.gaze_sequence or []
w = analysis.image_width
h = analysis.image_height
if score is None or not hotspots:
return insights
# 1. Attention Concentration
if score >= 60:
insights.append({
"type": "success",
"title": "Strong focal point",
"description": (
f"Attention is highly concentrated (score {score:.0f}/100). "
"Most viewers will fixate on the same areas — your design has a clear visual hierarchy."
),
})
elif score >= 30:
insights.append({
"type": "info",
"title": "Moderate attention spread",
"description": (
f"Attention is moderately distributed (score {score:.0f}/100). "
"Viewers will notice several areas. Consider whether your primary message "
"is prominent enough to stand out."
),
})
else:
insights.append({
"type": "warning",
"title": "Diffuse attention",
"description": (
f"Attention is widely spread (score {score:.0f}/100). "
"No single element dominates — viewers may struggle to find your key message. "
"Consider increasing contrast, size, or whitespace around your hero element."
),
})
# 2. Dominant Focal Point
if len(hotspots) >= 2:
top = hotspots[0]["intensity"]
second = hotspots[1]["intensity"]
if top > 0 and second > 0:
ratio = top / second
if ratio >= 3:
insights.append({
"type": "success",
"title": "Clear dominant element",
"description": (
f"The top hotspot is {ratio:.1f}x stronger than the second — "
"your design has one unmistakable focal point. This is ideal for "
"ads with a single hero product or CTA."
),
})
elif ratio >= 1.5:
insights.append({
"type": "info",
"title": "Moderate focal dominance",
"description": (
f"The top hotspot is {ratio:.1f}x stronger than the second. "
"There's a primary focus but competing elements may split attention."
),
})
else:
insights.append({
"type": "warning",
"title": "Competing focal points",
"description": (
"The top two hotspots have similar intensity — viewers' eyes will "
"bounce between them. If one element is your priority, consider "
"making it larger, brighter, or more isolated."
),
})
# 3. Gaze Entry Point
if gaze_seq:
first = gaze_seq[0]
x_pct = first.get("x_pct", first["x"] / w * 100 if w else 50)
y_pct = first.get("y_pct", first["y"] / h * 100 if h else 50)
if y_pct < 33:
v_zone = "top"
elif y_pct < 66:
v_zone = "middle"
else:
v_zone = "bottom"
if x_pct < 33:
h_zone = "left"
elif x_pct < 66:
h_zone = "center"
else:
h_zone = "right"
position = f"{v_zone}-{h_zone}"
insights.append({
"type": "info",
"title": f"First fixation: {position}",
"description": (
f"Viewers are predicted to look at the {position} area first "
f"({x_pct:.0f}% from left, {y_pct:.0f}% from top). "
"Place your most important message or brand element here for maximum impact."
),
})
# 4. Spatial Balance
if len(hotspots) >= 3:
quadrants = {"TL": 0, "TR": 0, "BL": 0, "BR": 0}
for hs in hotspots:
cx = hs.get("center_x", hs["x"])
cy = hs.get("center_y", hs["y"])
q_h = "L" if cx < w / 2 else "R"
q_v = "T" if cy < h / 2 else "B"
quadrants[q_v + q_h] += 1
max_q = max(quadrants.values())
max_q_name = {
"TL": "top-left", "TR": "top-right",
"BL": "bottom-left", "BR": "bottom-right",
}
dominant = [k for k, v in quadrants.items() if v == max_q]
if max_q >= 3:
zone = max_q_name[dominant[0]]
insights.append({
"type": "warning",
"title": f"Attention clusters in {zone}",
"description": (
f"{max_q} of {len(hotspots)} hotspots fall in the {zone} quadrant. "
"The opposite areas of your design may go largely unnoticed. "
"Consider rebalancing if key information is in the neglected zones."
),
})
# 5. Edge Risk
edge_threshold = 0.10 # 10% from edge
edge_hotspots = []
for hs in hotspots[:3]: # check top 3
cx = hs.get("center_x", hs["x"])
cy = hs.get("center_y", hs["y"])
near_edge = (
cx < w * edge_threshold
or cx > w * (1 - edge_threshold)
or cy < h * edge_threshold
or cy > h * (1 - edge_threshold)
)
if near_edge:
edge_hotspots.append(hs["rank"])
if edge_hotspots:
ranks = ", ".join(f"#{r}" for r in edge_hotspots)
insights.append({
"type": "warning",
"title": "Key attention near edge",
"description": (
f"Hotspot {ranks} {'is' if len(edge_hotspots) == 1 else 'are'} close to the "
"image edge. When printed or cropped, this attention area may be partially cut off. "
"Consider adding safe margins around critical content."
),
})
# 6. Attention Drop-off
if len(hotspots) >= 3:
intensities = [hs["intensity"] for hs in hotspots[:5]]
if intensities[0] > 0:
dropoff = intensities[-1] / intensities[0]
if dropoff < 0.2:
insights.append({
"type": "info",
"title": "Steep attention drop-off",
"description": (
f"Attention drops sharply from hotspot #1 ({intensities[0]:.0%}) to "
f"#{len(intensities)} ({intensities[-1]:.0%}). This means the design "
"has a very strong primary focus but secondary elements get little attention."
),
})
return insights

439
backend/app/services/report_generator.py Normal file
View file

@ -0,0 +1,439 @@
import io
import os
from datetime import datetime
from PIL import Image as PILImage
from reportlab.lib import colors
from reportlab.lib.pagesizes import A4
from reportlab.lib.styles import ParagraphStyle
from reportlab.lib.units import inch, mm
from reportlab.pdfbase import pdfmetrics
from reportlab.pdfbase.ttfonts import TTFont
from reportlab.platypus import (
Image,
PageBreak,
Paragraph,
SimpleDocTemplate,
Spacer,
Table,
TableStyle,
)
# ─── Colors ──────────────────────────────────────────
NAVY = colors.HexColor("#1a1a2e")
GOLD = colors.HexColor("#ffc407")
GOLD_LIGHT = colors.HexColor("#fff8e0")
LIGHT_GRAY = colors.HexColor("#f5f5f5")
MEDIUM_GRAY = colors.HexColor("#666666")
DARK_TEXT = colors.HexColor("#333333")
WHITE = colors.white
GREEN = colors.HexColor("#16a34a")
BLUE = colors.HexColor("#2563eb")
AMBER = colors.HexColor("#d97706")
PURPLE = colors.HexColor("#7c3aed")
# ─── Register Montserrat fonts ──────────────────────
FONT_DIR = os.path.join(os.path.dirname(__file__), "fonts")
pdfmetrics.registerFont(TTFont("Montserrat", os.path.join(FONT_DIR, "Montserrat-Regular.ttf")))
pdfmetrics.registerFont(TTFont("Montserrat-Bold", os.path.join(FONT_DIR, "Montserrat-Bold.ttf")))
pdfmetrics.registerFont(TTFont("Montserrat-SemiBold", os.path.join(FONT_DIR, "Montserrat-SemiBold.ttf")))
pdfmetrics.registerFontFamily(
"Montserrat",
normal="Montserrat",
bold="Montserrat-Bold",
)
def _bytes_to_image(data: bytes) -> io.BytesIO:
buf = io.BytesIO(data)
buf.seek(0)
return buf
def _make_styles():
title = ParagraphStyle(
"Title",
fontName="Montserrat-Bold",
fontSize=32,
textColor=NAVY,
spaceAfter=6,
leading=38,
)
subtitle = ParagraphStyle(
"Subtitle",
fontName="Montserrat",
fontSize=14,
textColor=GOLD,
spaceAfter=20,
leading=18,
)
heading = ParagraphStyle(
"Heading",
fontName="Montserrat-Bold",
fontSize=18,
textColor=NAVY,
spaceBefore=12,
spaceAfter=8,
leading=22,
)
subheading = ParagraphStyle(
"Subheading",
fontName="Montserrat-SemiBold",
fontSize=13,
textColor=NAVY,
spaceBefore=8,
spaceAfter=4,
leading=16,
)
body = ParagraphStyle(
"Body",
fontName="Montserrat",
fontSize=10,
textColor=DARK_TEXT,
spaceAfter=6,
leading=14,
)
body_small = ParagraphStyle(
"BodySmall",
fontName="Montserrat",
fontSize=9,
textColor=MEDIUM_GRAY,
spaceAfter=4,
leading=12,
)
meta_label = ParagraphStyle(
"MetaLabel",
fontName="Montserrat-SemiBold",
fontSize=10,
textColor=MEDIUM_GRAY,
spaceAfter=2,
)
meta_value = ParagraphStyle(
"MetaValue",
fontName="Montserrat-Bold",
fontSize=14,
textColor=NAVY,
spaceAfter=8,
)
footer = ParagraphStyle(
"Footer",
fontName="Montserrat",
fontSize=8,
textColor=MEDIUM_GRAY,
alignment=1, # center
)
return {
"title": title,
"subtitle": subtitle,
"heading": heading,
"subheading": subheading,
"body": body,
"body_small": body_small,
"meta_label": meta_label,
"meta_value": meta_value,
"footer": footer,
}
def _insight_type_label(t: str) -> tuple[str, colors.Color]:
return {
"success": ("STRENGTH", GREEN),
"warning": ("ATTENTION", AMBER),
"info": ("INSIGHT", BLUE),
}.get(t, ("INSIGHT", BLUE))
def _build_insight_table(insights: list[dict], styles: dict, is_ai: bool = False) -> list:
"""Build styled insight rows as ReportLab elements."""
elements = []
for insight in insights:
label_text, label_color = _insight_type_label(insight["type"])
accent = PURPLE if is_ai else label_color
# Build a mini table for each insight card
badge = f'<font color="#{accent.hexval()[2:]}" size="7"><b>{label_text}</b></font>'
title = f'<font name="Montserrat-SemiBold" size="10" color="#1a1a2e">{insight["title"]}</font>'
desc = f'<font name="Montserrat" size="9" color="#555555">{insight["description"]}</font>'
content = Paragraph(f"{badge}<br/>{title}<br/>{desc}", styles["body"])
t = Table(
[[content]],
colWidths=[170 * mm],
)
t.setStyle(TableStyle([
("BACKGROUND", (0, 0), (-1, -1), GOLD_LIGHT if is_ai else LIGHT_GRAY),
("LEFTPADDING", (0, 0), (-1, -1), 10),
("RIGHTPADDING", (0, 0), (-1, -1), 10),
("TOPPADDING", (0, 0), (-1, -1), 8),
("BOTTOMPADDING", (0, 0), (-1, -1), 8),
("ROUNDEDCORNERS", [4, 4, 4, 4]),
("LINEBEFOREDECORCOLOR", (0, 0), (0, -1), accent),
# Left accent border
("LINEBEFORE", (0, 0), (0, -1), 3, accent),
]))
elements.append(t)
elements.append(Spacer(1, 4))
return elements
def generate_report(
analysis,
original_image: bytes,
heatmap_image: bytes,
gaze_image: bytes,
aois: list,
rule_insights: list[dict] | None = None,
ai_insights: list[dict] | None = None,
ai_cost_usd: float | None = None,
) -> bytes:
"""Generate a professional PDF report for an analysis."""
buffer = io.BytesIO()
doc = SimpleDocTemplate(
buffer,
pagesize=A4,
topMargin=20 * mm,
bottomMargin=20 * mm,
leftMargin=15 * mm,
rightMargin=15 * mm,
)
styles = _make_styles()
elements = []
# ─── Cover Page ──────────────────────────────────────────
elements.append(Spacer(1, 50))
elements.append(Paragraph("OliVAS", styles["title"]))
elements.append(Paragraph("Visual Attention Analysis Report", styles["subtitle"]))
elements.append(Spacer(1, 8))
# Gold divider line
divider = Table([[""]], colWidths=[60 * mm], rowHeights=[2])
divider.setStyle(TableStyle([
("BACKGROUND", (0, 0), (-1, -1), GOLD),
("LEFTPADDING", (0, 0), (-1, -1), 0),
("RIGHTPADDING", (0, 0), (-1, -1), 0),
("TOPPADDING", (0, 0), (-1, -1), 0),
("BOTTOMPADDING", (0, 0), (-1, -1), 0),
]))
elements.append(divider)
elements.append(Spacer(1, 20))
# Metadata grid
meta_data = [
["ANALYSIS", "MODEL", "DATE", "IMAGE SIZE"],
[
analysis.name,
analysis.model_used.replace("_", " ").title(),
datetime.now().strftime("%B %d, %Y"),
f"{analysis.image_width} x {analysis.image_height}",
],
]
meta_table = Table(meta_data, colWidths=[45 * mm] * 4)
meta_table.setStyle(TableStyle([
("FONTNAME", (0, 0), (-1, 0), "Montserrat-SemiBold"),
("FONTSIZE", (0, 0), (-1, 0), 8),
("TEXTCOLOR", (0, 0), (-1, 0), MEDIUM_GRAY),
("FONTNAME", (0, 1), (-1, 1), "Montserrat-Bold"),
("FONTSIZE", (0, 1), (-1, 1), 11),
("TEXTCOLOR", (0, 1), (-1, 1), NAVY),
("TOPPADDING", (0, 0), (-1, -1), 4),
("BOTTOMPADDING", (0, 0), (-1, -1), 4),
("VALIGN", (0, 0), (-1, -1), "TOP"),
]))
elements.append(meta_table)
elements.append(Spacer(1, 10))
# Score badge
if analysis.overall_score is not None:
score = analysis.overall_score
score_color = GREEN if score >= 60 else (AMBER if score >= 30 else colors.HexColor("#ef4444"))
score_text = f'<font name="Montserrat-SemiBold" size="9" color="#666666">ATTENTION SCORE</font><br/><font name="Montserrat-Bold" size="28" color="#{score_color.hexval()[2:]}">{score:.0f}</font><font name="Montserrat" size="12" color="#999999"> / 100</font>'
elements.append(Paragraph(score_text, styles["body"]))
elements.append(Spacer(1, 10))
# Thumbnail on cover
max_width = 160 * mm
img = PILImage.open(io.BytesIO(original_image))
aspect = img.width / img.height
img_width = min(max_width, 130 * mm)
img_height = img_width / aspect
if img_height > 90 * mm:
img_height = 90 * mm
img_width = img_height * aspect
elements.append(Image(_bytes_to_image(original_image), width=img_width, height=img_height))
elements.append(PageBreak())
# ─── Heatmap Page ────────────────────────────────────────
elements.append(Paragraph("Attention Heatmap", styles["heading"]))
elements.append(
Paragraph(
"Areas highlighted in warm colors (red/yellow) indicate high predicted "
"attention during the first 3-5 seconds of viewing. Cool colors (blue) "
"indicate lower attention probability.",
styles["body"],
)
)
elements.append(Spacer(1, 8))
elements.append(Image(_bytes_to_image(heatmap_image), width=img_width, height=img_height))
elements.append(PageBreak())
# ─── Gaze Sequence Page ──────────────────────────────────
elements.append(Paragraph("Predicted Gaze Sequence", styles["heading"]))
elements.append(
Paragraph(
"Numbered points show the predicted order in which viewers will "
"fixate on different areas of the design. Point #1 is where the eye "
"lands first.",
styles["body"],
)
)
elements.append(Spacer(1, 8))
elements.append(Image(_bytes_to_image(gaze_image), width=img_width, height=img_height))
if analysis.gaze_sequence:
elements.append(Spacer(1, 10))
elements.append(Paragraph("Fixation Details", styles["subheading"]))
table_data = [["#", "Position", "From Left", "From Top", "Probability"]]
for point in analysis.gaze_sequence:
table_data.append([
str(point["rank"]),
f"({point['x']}, {point['y']})",
f"{point['x_pct']:.1f}%",
f"{point['y_pct']:.1f}%",
f"{point['probability']:.1%}",
])
t = Table(table_data, colWidths=[25, 80, 60, 60, 70])
t.setStyle(TableStyle([
("BACKGROUND", (0, 0), (-1, 0), NAVY),
("TEXTCOLOR", (0, 0), (-1, 0), WHITE),
("FONTNAME", (0, 0), (-1, 0), "Montserrat-SemiBold"),
("FONTNAME", (0, 1), (-1, -1), "Montserrat"),
("FONTSIZE", (0, 0), (-1, -1), 9),
("GRID", (0, 0), (-1, -1), 0.5, colors.HexColor("#dddddd")),
("ROWBACKGROUNDS", (0, 1), (-1, -1), [WHITE, LIGHT_GRAY]),
("ALIGN", (0, 0), (-1, -1), "CENTER"),
("TOPPADDING", (0, 0), (-1, -1), 5),
("BOTTOMPADDING", (0, 0), (-1, -1), 5),
]))
elements.append(t)
# ─── Hotspots Page ───────────────────────────────────────
if analysis.hotspots:
elements.append(PageBreak())
elements.append(Paragraph("Attention Hotspots", styles["heading"]))
elements.append(
Paragraph(
"The top regions ranked by predicted attention intensity. Higher rank "
"means more predicted visual attention.",
styles["body"],
)
)
elements.append(Spacer(1, 8))
hs_data = [["Rank", "Position (x, y)", "Intensity"]]
for hs in analysis.hotspots:
cx = hs.get("center_x", hs["x"])
cy = hs.get("center_y", hs["y"])
hs_data.append([
f"#{hs['rank']}",
f"({cx}, {cy})",
f"{hs['intensity']:.2%}",
])
t = Table(hs_data, colWidths=[50, 120, 80])
t.setStyle(TableStyle([
("BACKGROUND", (0, 0), (-1, 0), NAVY),
("TEXTCOLOR", (0, 0), (-1, 0), WHITE),
("FONTNAME", (0, 0), (-1, 0), "Montserrat-SemiBold"),
("FONTNAME", (0, 1), (-1, -1), "Montserrat"),
("FONTSIZE", (0, 0), (-1, -1), 9),
("GRID", (0, 0), (-1, -1), 0.5, colors.HexColor("#dddddd")),
("ROWBACKGROUNDS", (0, 1), (-1, -1), [WHITE, LIGHT_GRAY]),
("ALIGN", (0, 0), (-1, -1), "CENTER"),
("TOPPADDING", (0, 0), (-1, -1), 5),
("BOTTOMPADDING", (0, 0), (-1, -1), 5),
]))
elements.append(t)
# ─── Insights Page ───────────────────────────────────────
has_rule = rule_insights and len(rule_insights) > 0
has_ai = ai_insights and len(ai_insights) > 0
if has_rule or has_ai:
elements.append(PageBreak())
elements.append(Paragraph("Analysis Insights", styles["heading"]))
if has_rule:
elements.append(Paragraph("Rule-Based Insights", styles["subheading"]))
elements.append(
Paragraph(
"Automatically generated observations based on the attention metrics.",
styles["body_small"],
)
)
elements.append(Spacer(1, 6))
elements.extend(_build_insight_table(rule_insights, styles, is_ai=False))
elements.append(Spacer(1, 12))
if has_ai:
elements.append(Paragraph("AI Design Analysis", styles["subheading"]))
ai_note = "Generated by Claude Sonnet 4.6 — context-aware design recommendations."
if ai_cost_usd is not None:
ai_note += f" (Cost: ${ai_cost_usd:.4f})"
elements.append(Paragraph(ai_note, styles["body_small"]))
elements.append(Spacer(1, 6))
elements.extend(_build_insight_table(ai_insights, styles, is_ai=True))
# ─── AOI Page (if any) ───────────────────────────────────
if aois:
elements.append(PageBreak())
elements.append(Paragraph("Areas of Interest", styles["heading"]))
elements.append(
Paragraph(
"User-defined regions analyzed for predicted attention capture. "
"Density > 1.0 means the region captures more attention than its "
"size would suggest.",
styles["body"],
)
)
elements.append(Spacer(1, 8))
aoi_data = [["Region", "Attention %", "Area %", "Density"]]
for aoi in aois:
aoi_data.append([
aoi.label,
f"{aoi.attention_pct:.1f}%",
f"{aoi.area_pct:.1f}%",
f"{aoi.attention_density:.2f}x",
])
t = Table(aoi_data, colWidths=[100, 70, 70, 70])
t.setStyle(TableStyle([
("BACKGROUND", (0, 0), (-1, 0), NAVY),
("TEXTCOLOR", (0, 0), (-1, 0), WHITE),
("FONTNAME", (0, 0), (-1, 0), "Montserrat-SemiBold"),
("FONTNAME", (0, 1), (-1, -1), "Montserrat"),
("FONTSIZE", (0, 0), (-1, -1), 9),
("GRID", (0, 0), (-1, -1), 0.5, colors.HexColor("#dddddd")),
("ROWBACKGROUNDS", (0, 1), (-1, -1), [WHITE, LIGHT_GRAY]),
("ALIGN", (1, 0), (-1, -1), "CENTER"),
("TOPPADDING", (0, 0), (-1, -1), 5),
("BOTTOMPADDING", (0, 0), (-1, -1), 5),
]))
elements.append(t)
# ─── Footer ──────────────────────────────────────────────
elements.append(Spacer(1, 30))
elements.append(
Paragraph(
"Generated by OliVAS — Open-Source Visual Attention Software by OLIVER",
styles["footer"],
)
)
doc.build(elements)
return buffer.getvalue()

0
backend/app/services/saliency/__init__.py Normal file
View file

19
backend/app/services/saliency/base.py Normal file
View file

@ -0,0 +1,19 @@
from abc import ABC, abstractmethod
import numpy as np
from PIL import Image
class BaseSaliencyModel(ABC):
@abstractmethod
def load(self) -> None:
...
@abstractmethod
def predict(self, image: Image.Image) -> np.ndarray:
"""Returns 2D array (H, W) with values in [0, 1]."""
...
@abstractmethod
def get_name(self) -> str:
...

93
backend/app/services/saliency/deepgaze.py Normal file
View file

@ -0,0 +1,93 @@
import logging
import numpy as np
import torch
from PIL import Image
from scipy.ndimage import zoom as scipy_zoom
from scipy.special import logsumexp
from app.services.saliency.base import BaseSaliencyModel
logger = logging.getLogger("olivas.deepgaze")
# Map variant keys to deepgaze_pytorch classes
VARIANT_MAP = {
"I": ("DeepGazeI", "DeepGaze I"),
"IIE": ("DeepGazeIIE", "DeepGaze IIE"),
"III": ("DeepGazeIII", "DeepGaze III"),
}
class DeepGazeModel(BaseSaliencyModel):
"""Unified wrapper for all DeepGaze model variants (I, IIE, III)."""
def __init__(self, variant: str = "IIE", device: str = "cpu"):
if variant not in VARIANT_MAP:
raise ValueError(f"Unknown DeepGaze variant: {variant}. Choose from {list(VARIANT_MAP.keys())}")
self.variant = variant
self.class_name, self.display_name = VARIANT_MAP[variant]
self.device = torch.device(device)
self.model = None
self.centerbias_template = None
def load(self) -> None:
import deepgaze_pytorch
logger.info(f"Loading {self.display_name} on {self.device}...")
model_cls = getattr(deepgaze_pytorch, self.class_name)
self.model = model_cls(pretrained=True).to(self.device)
self.model.eval()
self._create_default_centerbias()
logger.info(f"{self.display_name} loaded successfully")
def _create_default_centerbias(self):
"""Create a generic center bias prior (Gaussian centered)."""
size = 1024
x = np.linspace(-1, 1, size)
y = np.linspace(-1, 1, size)
xx, yy = np.meshgrid(x, y)
self.centerbias_template = -0.5 * (xx**2 + yy**2) / 0.5**2
def predict(self, image: Image.Image) -> np.ndarray:
img_np = np.array(image.convert("RGB"))
h, w = img_np.shape[:2]
# Prepare image tensor [1, C, H, W]
image_tensor = (
torch.tensor([img_np.transpose(2, 0, 1)])
.float()
.to(self.device)
)
# Prepare centerbias
cb = scipy_zoom(
self.centerbias_template,
(h / self.centerbias_template.shape[0], w / self.centerbias_template.shape[1]),
order=0,
)
cb -= logsumexp(cb)
centerbias_tensor = (
torch.tensor([cb]).float().to(self.device)
)
with torch.no_grad():
log_density = self.model(image_tensor, centerbias_tensor)
saliency = torch.exp(log_density).cpu().numpy().squeeze()
# Normalize to [0, 1]
sal_min, sal_max = saliency.min(), saliency.max()
if sal_max - sal_min > 1e-10:
saliency = (saliency - sal_min) / (sal_max - sal_min)
else:
saliency = np.zeros_like(saliency)
return saliency
def get_name(self) -> str:
return self.display_name
# Backwards-compatible alias
DeepGazeIIEModel = lambda device="cpu": DeepGazeModel(variant="IIE", device=device)

55
backend/app/services/saliency/model_manager.py Normal file
View file

@ -0,0 +1,55 @@
import logging
import numpy as np
from PIL import Image
from app.services.saliency.base import BaseSaliencyModel
logger = logging.getLogger("olivas.model_manager")
class ModelManager:
def __init__(self):
self.models: dict[str, BaseSaliencyModel] = {}
self.default_model = "deepgaze_iie"
def load_models(self, device: str = "cpu") -> None:
from app.services.saliency.deepgaze import DeepGazeModel
variants = [
("deepgaze_i", "I"),
("deepgaze_iie", "IIE"),
("deepgaze_iii", "III"),
]
for key, variant in variants:
try:
model = DeepGazeModel(variant=variant, device=device)
model.load()
self.models[key] = model
logger.info(f"Loaded {model.get_name()}")
except Exception as e:
logger.warning(f"Failed to load DeepGaze {variant}: {e}")
def predict(self, image: Image.Image, model_name: str | None = None) -> np.ndarray:
name = model_name or self.default_model
if name not in self.models:
raise RuntimeError(f"Model '{name}' not loaded. Available: {list(self.models.keys())}")
return self.models[name].predict(image)
def list_models(self) -> list[dict]:
return [
{"id": key, "name": model.get_name()} for key, model in self.models.items()
]
def cleanup(self):
self.models.clear()
try:
import torch
if torch.cuda.is_available():
torch.cuda.empty_cache()
except ImportError:
pass
model_manager = ModelManager()

44
backend/app/services/storage.py Normal file
View file

@ -0,0 +1,44 @@
import os
from pathlib import Path
import aiofiles
from app.config import settings
class LocalStorage:
def __init__(self):
self.base_dir = Path(settings.UPLOAD_DIR)
self.base_dir.mkdir(parents=True, exist_ok=True)
def analysis_dir(self, analysis_id: str) -> Path:
path = self.base_dir / analysis_id
path.mkdir(parents=True, exist_ok=True)
return path
async def save_bytes(self, data: bytes, analysis_id: str, filename: str) -> str:
dir_path = self.analysis_dir(analysis_id)
file_path = dir_path / filename
async with aiofiles.open(file_path, "wb") as f:
await f.write(data)
return str(file_path)
async def load_bytes(self, analysis_id: str, filename: str) -> bytes:
file_path = self.analysis_dir(analysis_id) / filename
async with aiofiles.open(file_path, "rb") as f:
return await f.read()
def get_path(self, analysis_id: str, filename: str) -> Path:
return self.analysis_dir(analysis_id) / filename
def exists(self, analysis_id: str, filename: str) -> bool:
return (self.analysis_dir(analysis_id) / filename).exists()
async def delete_analysis(self, analysis_id: str) -> None:
import shutil
dir_path = self.base_dir / analysis_id
if dir_path.exists():
shutil.rmtree(dir_path)
storage = LocalStorage()

45
backend/pyproject.toml Normal file
View file

@ -0,0 +1,45 @@
[project]
name = "olivas-backend"
version = "0.1.0"
description = "OliVAS - Open-Source Visual Attention Software backend"
requires-python = ">=3.12"
dependencies = [
"fastapi>=0.115.0",
"uvicorn[standard]>=0.30.0",
"python-multipart>=0.0.9",
"sqlalchemy[asyncio]>=2.0",
"alembic>=1.13",
"asyncpg>=0.29",
"pydantic>=2.0",
"pydantic-settings>=2.0",
"pillow>=10.0",
"numpy>=1.26",
"torch>=2.1",
"torchvision>=0.16",
"scipy>=1.11",
"matplotlib>=3.8",
"scikit-image>=0.22",
"reportlab>=4.0",
"aiofiles>=23.0",
"anthropic>=0.40",
]
[project.optional-dependencies]
dev = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"httpx>=0.27",
"ruff>=0.4",
]
[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"
[tool.ruff]
line-length = 100
target-version = "py312"
[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]

0
backend/tests/__init__.py Normal file
View file

51
backend/tests/conftest.py Normal file
View file

@ -0,0 +1,51 @@
import asyncio
import pytest
import pytest_asyncio
from httpx import ASGITransport, AsyncClient
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from app.main import app
from app.models.base import Base
from app.db.session import get_db
# Use SQLite for tests
TEST_DATABASE_URL = "sqlite+aiosqlite:///./test.db"
@pytest.fixture(scope="session")
def event_loop():
loop = asyncio.new_event_loop()
yield loop
loop.close()
@pytest_asyncio.fixture
async def db_engine():
engine = create_async_engine(TEST_DATABASE_URL, echo=False)
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
yield engine
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.drop_all)
await engine.dispose()
@pytest_asyncio.fixture
async def db_session(db_engine):
session_factory = async_sessionmaker(db_engine, class_=AsyncSession, expire_on_commit=False)
async with session_factory() as session:
yield session
@pytest_asyncio.fixture
async def client(db_session):
async def override_get_db():
yield db_session
app.dependency_overrides[get_db] = override_get_db
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as ac:
yield ac
app.dependency_overrides.clear()

94
backend/tests/test_analysis.py Normal file
View file

@ -0,0 +1,94 @@
import numpy as np
import pytest
from app.services.gaze_sequence import extract_gaze_sequence
from app.services.aoi_analysis import compute_aoi_attention
from app.services.heatmap import generate_heatmap_overlay, generate_standalone_heatmap
from app.services.image_processing import prepare_for_inference, upscale_saliency
from PIL import Image
class TestGazeSequence:
def test_extracts_correct_number_of_fixations(self):
saliency = np.random.rand(100, 100)
result = extract_gaze_sequence(saliency, num_fixations=5)
assert len(result) == 5
def test_fixations_are_ranked(self):
saliency = np.random.rand(100, 100)
result = extract_gaze_sequence(saliency, num_fixations=3)
ranks = [p["rank"] for p in result]
assert ranks == [1, 2, 3]
def test_first_fixation_is_at_peak(self):
saliency = np.zeros((100, 100))
saliency[50, 75] = 1.0 # Set a clear peak
result = extract_gaze_sequence(saliency, num_fixations=1)
assert result[0]["x"] == 75
assert result[0]["y"] == 50
def test_coordinates_have_percentages(self):
saliency = np.random.rand(200, 300)
result = extract_gaze_sequence(saliency, num_fixations=1)
assert 0 <= result[0]["x_pct"] <= 100
assert 0 <= result[0]["y_pct"] <= 100
class TestAOIAnalysis:
def test_full_image_aoi_gets_100_percent(self):
saliency = np.ones((100, 100))
regions = [{"x": 0, "y": 0, "width": 100, "height": 100}]
result = compute_aoi_attention(saliency, regions)
assert abs(result[0]["attention_pct"] - 100.0) < 0.1
def test_half_image_aoi(self):
saliency = np.ones((100, 100))
regions = [{"x": 0, "y": 0, "width": 50, "height": 100}]
result = compute_aoi_attention(saliency, regions)
assert abs(result[0]["attention_pct"] - 50.0) < 1.0
def test_density_calculation(self):
saliency = np.zeros((100, 100))
saliency[0:10, 0:10] = 1.0 # High saliency in small region
regions = [{"x": 0, "y": 0, "width": 10, "height": 10}]
result = compute_aoi_attention(saliency, regions)
assert result[0]["attention_density"] > 1.0
def test_zero_saliency(self):
saliency = np.zeros((100, 100))
regions = [{"x": 0, "y": 0, "width": 50, "height": 50}]
result = compute_aoi_attention(saliency, regions)
assert result[0]["attention_pct"] == 0.0
class TestHeatmap:
def test_generates_overlay(self):
img = Image.new("RGB", (100, 100), color="white")
saliency = np.random.rand(100, 100)
result = generate_heatmap_overlay(img, saliency)
assert result.size == (100, 100)
assert result.mode == "RGB"
def test_generates_standalone(self):
saliency = np.random.rand(100, 100)
result = generate_standalone_heatmap(saliency)
assert result.size == (100, 100)
class TestImageProcessing:
def test_resize_large_image(self):
img = Image.new("RGB", (2000, 1000))
resized, scale = prepare_for_inference(img)
assert max(resized.size) <= 1024
assert scale < 1.0
def test_no_resize_small_image(self):
img = Image.new("RGB", (500, 300))
resized, scale = prepare_for_inference(img)
assert resized.size == (500, 300)
assert scale == 1.0
def test_upscale_saliency(self):
saliency = np.random.rand(50, 50)
result = upscale_saliency(saliency, 200, 300)
assert result.shape == (200, 300)

16
docker-compose.dev.yml Normal file
View file

@ -0,0 +1,16 @@
services:
backend:
build:
context: ./backend
command: uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
volumes:
- ./backend:/app
- uploads:/app/data/uploads
frontend:
build:
context: ./frontend
command: npm run dev -- --host 0.0.0.0
volumes:
- ./frontend:/app
- /app/node_modules

42
docker-compose.yml Normal file
View file

@ -0,0 +1,42 @@
services:
postgres:
image: postgres:16-alpine
ports:
- "5453:5432"
environment:
POSTGRES_USER: olivas
POSTGRES_PASSWORD: olivas
POSTGRES_DB: olivas
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U olivas"]
interval: 5s
timeout: 5s
retries: 5
backend:
build: ./backend
ports:
- "8000:8000"
environment:
DATABASE_URL: postgresql+asyncpg://olivas:olivas@postgres:5432/olivas
UPLOAD_DIR: /app/data/uploads
DEVICE: auto
CORS_ORIGINS: http://localhost:1577
volumes:
- uploads:/app/data/uploads
depends_on:
postgres:
condition: service_healthy
frontend:
build: ./frontend
ports:
- "1577:1577"
depends_on:
- backend
volumes:
pgdata:
uploads:

24
frontend/.gitignore vendored Normal file
View file

@ -0,0 +1,24 @@
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*
node_modules
dist
dist-ssr
*.local
# Editor directories and files
.vscode/*
!.vscode/extensions.json
.idea
.DS_Store
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?

13
frontend/Dockerfile Normal file
View file

@ -0,0 +1,13 @@
FROM node:20-alpine AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build
FROM nginx:alpine
COPY --from=build /app/dist /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 1577
CMD ["nginx", "-g", "daemon off;"]

73
frontend/README.md Normal file
View file

@ -0,0 +1,73 @@
# React + TypeScript + Vite
This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.
Currently, two official plugins are available:
- [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Babel](https://babeljs.io/) (or [oxc](https://oxc.rs) when used in [rolldown-vite](https://vite.dev/guide/rolldown)) for Fast Refresh
- [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/) for Fast Refresh
## React Compiler
The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation).
## Expanding the ESLint configuration
If you are developing a production application, we recommend updating the configuration to enable type-aware lint rules:
```js
export default defineConfig([
globalIgnores(['dist']),
{
files: ['**/*.{ts,tsx}'],
extends: [
// Other configs...
// Remove tseslint.configs.recommended and replace with this
tseslint.configs.recommendedTypeChecked,
// Alternatively, use this for stricter rules
tseslint.configs.strictTypeChecked,
// Optionally, add this for stylistic rules
tseslint.configs.stylisticTypeChecked,
// Other configs...
],
languageOptions: {
parserOptions: {
project: ['./tsconfig.node.json', './tsconfig.app.json'],
tsconfigRootDir: import.meta.dirname,
},
// other options...
},
},
])
```
You can also install [eslint-plugin-react-x](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-x) and [eslint-plugin-react-dom](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-dom) for React-specific lint rules:
```js
// eslint.config.js
import reactX from 'eslint-plugin-react-x'
import reactDom from 'eslint-plugin-react-dom'
export default defineConfig([
globalIgnores(['dist']),
{
files: ['**/*.{ts,tsx}'],
extends: [
// Other configs...
// Enable lint rules for React
reactX.configs['recommended-typescript'],
// Enable lint rules for React DOM
reactDom.configs.recommended,
],
languageOptions: {
parserOptions: {
project: ['./tsconfig.node.json', './tsconfig.app.json'],
tsconfigRootDir: import.meta.dirname,
},
// other options...
},
},
])
```

23
frontend/eslint.config.js Normal file
View file

@ -0,0 +1,23 @@
import js from '@eslint/js'
import globals from 'globals'
import reactHooks from 'eslint-plugin-react-hooks'
import reactRefresh from 'eslint-plugin-react-refresh'
import tseslint from 'typescript-eslint'
import { defineConfig, globalIgnores } from 'eslint/config'
export default defineConfig([
globalIgnores(['dist']),
{
files: ['**/*.{ts,tsx}'],
extends: [
js.configs.recommended,
tseslint.configs.recommended,
reactHooks.configs.flat.recommended,
reactRefresh.configs.vite,
],
languageOptions: {
ecmaVersion: 2020,
globals: globals.browser,
},
},
])

16
frontend/index.html Normal file
View file

@ -0,0 +1,16 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link href="https://fonts.googleapis.com/css2?family=Montserrat:ital,wght@0,300;0,400;0,500;0,600;0,700;0,800;1,400&display=swap" rel="stylesheet" />
<title>OliVAS</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>

18
frontend/nginx.conf Normal file
View file

@ -0,0 +1,18 @@
server {
listen 1577;
server_name localhost;
root /usr/share/nginx/html;
index index.html;
location /api {
proxy_pass http://backend:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 50M;
}
location / {
try_files $uri $uri/ /index.html;
}
}

4364
frontend/package-lock.json generated Normal file
View file

File diff suppressed because it is too large Load diff

39
frontend/package.json Normal file
View file

@ -0,0 +1,39 @@
{
"name": "frontend",
"private": true,
"version": "0.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc -b && vite build",
"lint": "eslint .",
"preview": "vite preview"
},
"dependencies": {
"@tailwindcss/vite": "^4.2.1",
"@tanstack/react-query": "^5.90.21",
"axios": "^1.13.5",
"konva": "^10.2.0",
"react": "^19.2.0",
"react-dom": "^19.2.0",
"react-dropzone": "^15.0.0",
"react-konva": "^19.2.2",
"react-router-dom": "^7.13.1",
"tailwindcss": "^4.2.1",
"zustand": "^5.0.11"
},
"devDependencies": {
"@eslint/js": "^9.39.1",
"@types/node": "^24.10.1",
"@types/react": "^19.2.7",
"@types/react-dom": "^19.2.3",
"@vitejs/plugin-react": "^5.1.1",
"eslint": "^9.39.1",
"eslint-plugin-react-hooks": "^7.0.1",
"eslint-plugin-react-refresh": "^0.4.24",
"globals": "^16.5.0",
"typescript": "~5.9.3",
"typescript-eslint": "^8.48.0",
"vite": "^7.3.1"
}
}

1
frontend/public/vite.svg Normal file
View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="31.88" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 257"><defs><linearGradient id="IconifyId1813088fe1fbc01fb466" x1="-.828%" x2="57.636%" y1="7.652%" y2="78.411%"><stop offset="0%" stop-color="#41D1FF"></stop><stop offset="100%" stop-color="#BD34FE"></stop></linearGradient><linearGradient id="IconifyId1813088fe1fbc01fb467" x1="43.376%" x2="50.316%" y1="2.242%" y2="89.03%"><stop offset="0%" stop-color="#FFEA83"></stop><stop offset="8.333%" stop-color="#FFDD35"></stop><stop offset="100%" stop-color="#FFA800"></stop></linearGradient></defs><path fill="url(#IconifyId1813088fe1fbc01fb466)" d="M255.153 37.938L134.897 252.976c-2.483 4.44-8.862 4.466-11.382.048L.875 37.958c-2.746-4.814 1.371-10.646 6.827-9.67l120.385 21.517a6.537 6.537 0 0 0 2.322-.004l117.867-21.483c5.438-.991 9.574 4.796 6.877 9.62Z"></path><path fill="url(#IconifyId1813088fe1fbc01fb467)" d="M185.432.063L96.44 17.501a3.268 3.268 0 0 0-2.634 3.014l-5.474 92.456a3.268 3.268 0 0 0 3.997 3.378l24.777-5.718c2.318-.535 4.413 1.507 3.936 3.838l-7.361 36.047c-.495 2.426 1.782 4.5 4.151 3.78l15.304-4.649c2.372-.72 4.652 1.36 4.15 3.788l-11.698 56.621c-.732 3.542 3.979 5.473 5.943 2.437l1.313-2.028l72.516-144.72c1.215-2.423-.88-5.186-3.54-4.672l-25.505 4.922c-2.396.462-4.435-1.77-3.759-4.114l16.646-57.705c.677-2.35-1.37-4.583-3.769-4.113Z"></path></svg>
Side by side

After

Width:  |  Height:  |  Size: 1.5 KiB

27
frontend/src/App.tsx Normal file
View file

@ -0,0 +1,27 @@
import { Routes, Route } from "react-router-dom";
import AppLayout from "./components/layout/AppLayout";
import Dashboard from "./pages/Dashboard";
import ProjectDetail from "./pages/ProjectDetail";
import NewAnalysis from "./pages/NewAnalysis";
import AnalysisView from "./pages/AnalysisView";
import ComparisonView from "./pages/ComparisonView";
import Help from "./pages/Help";
import About from "./pages/About";
function App() {
return (
<Routes>
<Route element={<AppLayout />}>
<Route path="/" element={<Dashboard />} />
<Route path="/projects/:projectId" element={<ProjectDetail />} />
<Route path="/analyze" element={<NewAnalysis />} />
<Route path="/analyze/:analysisId" element={<AnalysisView />} />
<Route path="/compare/:comparisonId" element={<ComparisonView />} />
<Route path="/help" element={<Help />} />
<Route path="/about" element={<About />} />
</Route>
</Routes>
);
}
export default App;

83
frontend/src/api/analysis.ts Normal file
View file

@ -0,0 +1,83 @@
import client from "./client";
import type { AnalysisDetail, AnalysisStatus, AOIRegion, AOIResult, Insight } from "../types/analysis";
export async function createAnalysis(
projectId: string,
file: File,
name?: string,
model?: string,
): Promise<AnalysisDetail> {
const formData = new FormData();
formData.append("file", file);
if (name) formData.append("name", name);
if (model) formData.append("model", model);
const { data } = await client.post(`/projects/${projectId}/analyses`, formData, {
headers: { "Content-Type": "multipart/form-data" },
});
return data;
}
export async function getAnalysis(id: string): Promise<AnalysisDetail> {
const { data } = await client.get(`/analyses/${id}`);
return data;
}
export async function getAnalysisStatus(id: string): Promise<AnalysisStatus> {
const { data } = await client.get(`/analyses/${id}/status`);
return data;
}
export async function deleteAnalysis(id: string): Promise<void> {
await client.delete(`/analyses/${id}`);
}
export function getAnalysisImageUrl(
id: string,
type: "original" | "saliency-raw" | "heatmap" | "heatmap-standalone" | "gaze-sequence" | "thumbnail",
): string {
return `/api/analyses/${id}/images/${type}`;
}
export async function createAOIs(
analysisId: string,
regions: AOIRegion[],
): Promise<AOIResult[]> {
const { data } = await client.post(`/analyses/${analysisId}/aois`, {
regions,
});
return data;
}
export async function getAOIs(analysisId: string): Promise<AOIResult[]> {
const { data } = await client.get(`/analyses/${analysisId}/aois`);
return data;
}
export async function checkAIInsightsAvailable(): Promise<boolean> {
try {
const { data } = await client.get("/analyses/ai-insights-available");
return data.available;
} catch {
return false;
}
}
export interface AIInsightsResponse {
insights: Insight[];
cost_usd: number;
input_tokens: number;
output_tokens: number;
}
export async function generateAIInsights(analysisId: string): Promise<AIInsightsResponse> {
const { data } = await client.post(`/analyses/${analysisId}/ai-insights`);
return data;
}
export async function deleteAOI(
analysisId: string,
aoiId: string,
): Promise<void> {
await client.delete(`/analyses/${analysisId}/aois/${aoiId}`);
}

25
frontend/src/api/client.ts Normal file
View file

@ -0,0 +1,25 @@
import axios from "axios";
const client = axios.create({
baseURL: "/api",
headers: {
"Content-Type": "application/json",
},
});
client.interceptors.request.use((config) => {
config.headers["X-User-Id"] = "default";
return config;
});
client.interceptors.response.use(
(response) => response,
(error) => {
const message =
error.response?.data?.detail || error.message || "An error occurred";
console.error("[API Error]", message);
return Promise.reject(error);
},
);
export default client;

37
frontend/src/api/projects.ts Normal file
View file

@ -0,0 +1,37 @@
import client from "./client";
import type { Project, ProjectDetail, ProjectCreate } from "../types/project";
export async function createProject(
name: string,
description?: string,
): Promise<Project> {
const { data } = await client.post("/projects", { name, description });
return data;
}
export async function listProjects(
page: number = 1,
perPage: number = 20,
): Promise<Project[]> {
const { data } = await client.get("/projects", {
params: { page, per_page: perPage },
});
return data;
}
export async function getProject(id: string): Promise<ProjectDetail> {
const { data } = await client.get(`/projects/${id}`);
return data;
}
export async function updateProject(
id: string,
updates: Partial<ProjectCreate>,
): Promise<Project> {
const { data } = await client.put(`/projects/${id}`, updates);
return data;
}
export async function deleteProject(id: string): Promise<void> {
await client.delete(`/projects/${id}`);
}

23
frontend/src/api/reports.ts Normal file
View file

@ -0,0 +1,23 @@
import client from "./client";
export async function downloadReport(analysisId: string): Promise<void> {
const response = await client.get(`/analyses/${analysisId}/report`, {
responseType: "blob",
});
const blob = new Blob([response.data], { type: "application/pdf" });
const url = window.URL.createObjectURL(blob);
const link = document.createElement("a");
link.href = url;
const disposition = response.headers["content-disposition"];
const filename = disposition
? disposition.split("filename=")[1]?.replace(/"/g, "")
: `analysis-${analysisId}-report.pdf`;
link.download = filename;
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
window.URL.revokeObjectURL(url);
}

1
frontend/src/assets/react.svg Normal file
View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="35.93" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 228"><path fill="#00D8FF" d="M210.483 73.824a171.49 171.49 0 0 0-8.24-2.597c.465-1.9.893-3.777 1.273-5.621c6.238-30.281 2.16-54.676-11.769-62.708c-13.355-7.7-35.196.329-57.254 19.526a171.23 171.23 0 0 0-6.375 5.848a155.866 155.866 0 0 0-4.241-3.917C100.759 3.829 77.587-4.822 63.673 3.233C50.33 10.957 46.379 33.89 51.995 62.588a170.974 170.974 0 0 0 1.892 8.48c-3.28.932-6.445 1.924-9.474 2.98C17.309 83.498 0 98.307 0 113.668c0 15.865 18.582 31.778 46.812 41.427a145.52 145.52 0 0 0 6.921 2.165a167.467 167.467 0 0 0-2.01 9.138c-5.354 28.2-1.173 50.591 12.134 58.266c13.744 7.926 36.812-.22 59.273-19.855a145.567 145.567 0 0 0 5.342-4.923a168.064 168.064 0 0 0 6.92 6.314c21.758 18.722 43.246 26.282 56.54 18.586c13.731-7.949 18.194-32.003 12.4-61.268a145.016 145.016 0 0 0-1.535-6.842c1.62-.48 3.21-.974 4.76-1.488c29.348-9.723 48.443-25.443 48.443-41.52c0-15.417-17.868-30.326-45.517-39.844Zm-6.365 70.984c-1.4.463-2.836.91-4.3 1.345c-3.24-10.257-7.612-21.163-12.963-32.432c5.106-11 9.31-21.767 12.459-31.957c2.619.758 5.16 1.557 7.61 2.4c23.69 8.156 38.14 20.213 38.14 29.504c0 9.896-15.606 22.743-40.946 31.14Zm-10.514 20.834c2.562 12.94 2.927 24.64 1.23 33.787c-1.524 8.219-4.59 13.698-8.382 15.893c-8.067 4.67-25.32-1.4-43.927-17.412a156.726 156.726 0 0 1-6.437-5.87c7.214-7.889 14.423-17.06 21.459-27.246c12.376-1.098 24.068-2.894 34.671-5.345a134.17 134.17 0 0 1 1.386 6.193ZM87.276 214.515c-7.882 2.783-14.16 2.863-17.955.675c-8.075-4.657-11.432-22.636-6.853-46.752a156.923 156.923 0 0 1 1.869-8.499c10.486 2.32 22.093 3.988 34.498 4.994c7.084 9.967 14.501 19.128 21.976 27.15a134.668 134.668 0 0 1-4.877 4.492c-9.933 8.682-19.886 14.842-28.658 17.94ZM50.35 144.747c-12.483-4.267-22.792-9.812-29.858-15.863c-6.35-5.437-9.555-10.836-9.555-15.216c0-9.322 13.897-21.212 37.076-29.293c2.813-.98 5.757-1.905 8.812-2.773c3.204 10.42 7.406 21.315 12.477 32.332c-5.137 11.18-9.399 22.249-12.634 32.792a134.718 134.718 0 0 1-6.318-1.979Zm12.378-84.26c-4.811-24.587-1.616-43.134 6.425-47.789c8.564-4.958 27.502 2.111 47.463 19.835a144.318 144.318 0 0 1 3.841 3.545c-7.438 7.987-14.787 17.08-21.808 26.988c-12.04 1.116-23.565 2.908-34.161 5.309a160.342 160.342 0 0 1-1.76-7.887Zm110.427 27.268a347.8 347.8 0 0 0-7.785-12.803c8.168 1.033 15.994 2.404 23.343 4.08c-2.206 7.072-4.956 14.465-8.193 22.045a381.151 381.151 0 0 0-7.365-13.322Zm-45.032-43.861c5.044 5.465 10.096 11.566 15.065 18.186a322.04 322.04 0 0 0-30.257-.006c4.974-6.559 10.069-12.652 15.192-18.18ZM82.802 87.83a323.167 323.167 0 0 0-7.227 13.238c-3.184-7.553-5.909-14.98-8.134-22.152c7.304-1.634 15.093-2.97 23.209-3.984a321.524 321.524 0 0 0-7.848 12.897Zm8.081 65.352c-8.385-.936-16.291-2.203-23.593-3.793c2.26-7.3 5.045-14.885 8.298-22.6a321.187 321.187 0 0 0 7.257 13.246c2.594 4.48 5.28 8.868 8.038 13.147Zm37.542 31.03c-5.184-5.592-10.354-11.779-15.403-18.433c4.902.192 9.899.29 14.978.29c5.218 0 10.376-.117 15.453-.343c-4.985 6.774-10.018 12.97-15.028 18.486Zm52.198-57.817c3.422 7.8 6.306 15.345 8.596 22.52c-7.422 1.694-15.436 3.058-23.88 4.071a382.417 382.417 0 0 0 7.859-13.026a347.403 347.403 0 0 0 7.425-13.565Zm-16.898 8.101a358.557 358.557 0 0 1-12.281 19.815a329.4 329.4 0 0 1-23.444.823c-7.967 0-15.716-.248-23.178-.732a310.202 310.202 0 0 1-12.513-19.846h.001a307.41 307.41 0 0 1-10.923-20.627a310.278 310.278 0 0 1 10.89-20.637l-.001.001a307.318 307.318 0 0 1 12.413-19.761c7.613-.576 15.42-.876 23.31-.876H128c7.926 0 15.743.303 23.354.883a329.357 329.357 0 0 1 12.335 19.695a358.489 358.489 0 0 1 11.036 20.54a329.472 329.472 0 0 1-11 20.722Zm22.56-122.124c8.572 4.944 11.906 24.881 6.52 51.026c-.344 1.668-.73 3.367-1.15 5.09c-10.622-2.452-22.155-4.275-34.23-5.408c-7.034-10.017-14.323-19.124-21.64-27.008a160.789 160.789 0 0 1 5.888-5.4c18.9-16.447 36.564-22.941 44.612-18.3ZM128 90.808c12.625 0 22.86 10.235 22.86 22.86s-10.235 22.86-22.86 22.86s-22.86-10.235-22.86-22.86s10.235-22.86 22.86-22.86Z"></path></svg>
Side by side

After

Width:  |  Height:  |  Size: 4 KiB

95
frontend/src/components/analysis/GazeSequence.tsx Normal file
View file

@ -0,0 +1,95 @@
import type { GazePoint } from "../../types/analysis";
import { useAnalysisStore } from "../../stores/analysisStore";
import Card from "../common/Card";
interface GazeSequenceProps {
imageUrl: string;
gazePoints: GazePoint[];
}
export default function GazeSequence({
imageUrl,
gazePoints,
}: GazeSequenceProps) {
const zoom = useAnalysisStore((s) => s.zoom);
return (
<div className="space-y-4">
<div className="rounded-lg overflow-auto border border-gray-200">
<img
src={imageUrl}
alt="Gaze sequence visualization"
className="object-contain"
style={{ width: `${zoom * 100}%`, maxWidth: "none" }}
/>
</div>
{gazePoints.length > 0 && (
<Card>
<h3 className="text-sm font-semibold text-gray-700 uppercase tracking-wide mb-3">
Predicted Gaze Sequence
</h3>
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead>
<tr className="border-b border-gray-200">
<th className="text-left py-2 px-3 text-gray-500 font-medium">
Order
</th>
<th className="text-left py-2 px-3 text-gray-500 font-medium">
X
</th>
<th className="text-left py-2 px-3 text-gray-500 font-medium">
Y
</th>
<th className="text-left py-2 px-3 text-gray-500 font-medium">
Intensity
</th>
</tr>
</thead>
<tbody>
{gazePoints.map((point) => (
<tr
key={point.rank}
className="border-b border-gray-100 hover:bg-gray-50"
>
<td className="py-2 px-3">
<span
className="inline-flex items-center justify-center w-6 h-6 rounded-full text-white text-xs font-bold"
style={{ backgroundColor: "#ffc407" }}
>
{point.rank}
</span>
</td>
<td className="py-2 px-3 text-gray-700">
{Math.round(point.x)}
</td>
<td className="py-2 px-3 text-gray-700">
{Math.round(point.y)}
</td>
<td className="py-2 px-3">
<div className="flex items-center gap-2">
<div className="flex-1 bg-gray-200 rounded-full h-2 max-w-[100px]">
<div
className="h-2 rounded-full"
style={{
width: `${point.intensity * 100}%`,
backgroundColor: "#ffc407",
}}
/>
</div>
<span className="text-gray-500 text-xs">
{(point.intensity * 100).toFixed(0)}%
</span>
</div>
</td>
</tr>
))}
</tbody>
</table>
</div>
</Card>
)}
</div>
);
}

68
frontend/src/components/analysis/HeatmapControls.tsx Normal file
View file

@ -0,0 +1,68 @@
import { useAnalysisStore } from "../../stores/analysisStore";
import Card from "../common/Card";
const colormaps = ["jet", "viridis", "inferno", "magma", "plasma"];
export default function HeatmapControls() {
const { opacity, colormap, visible } = useAnalysisStore(
(s) => s.heatmapSettings,
);
const setOpacity = useAnalysisStore((s) => s.setHeatmapOpacity);
const setColormap = useAnalysisStore((s) => s.setHeatmapColormap);
const setVisible = useAnalysisStore((s) => s.setHeatmapVisible);
return (
<Card className="space-y-4">
<h3 className="text-sm font-semibold text-gray-700 uppercase tracking-wide">
Heatmap Controls
</h3>
<div className="flex items-center justify-between">
<label className="text-sm text-gray-600">Show Overlay</label>
<button
onClick={() => setVisible(!visible)}
className={`relative w-10 h-5 rounded-full transition-colors ${
visible ? "bg-[#ffc407]" : "bg-gray-300"
}`}
>
<span
className={`absolute top-0.5 left-0.5 w-4 h-4 bg-white rounded-full shadow transition-transform ${
visible ? "translate-x-5" : ""
}`}
/>
</button>
</div>
<div>
<label className="text-sm text-gray-600 block mb-1">
Opacity: {Math.round(opacity * 100)}%
</label>
<input
type="range"
min="0"
max="100"
value={Math.round(opacity * 100)}
onChange={(e) => setOpacity(Number(e.target.value) / 100)}
className="w-full accent-[#ffc407]"
disabled={!visible}
/>
</div>
<div>
<label className="text-sm text-gray-600 block mb-1">Colormap</label>
<select
value={colormap}
onChange={(e) => setColormap(e.target.value)}
disabled={!visible}
className="w-full border border-gray-300 rounded-lg px-3 py-2 text-sm bg-white focus:outline-none focus:ring-2 focus:ring-[#ffc407] focus:border-transparent disabled:opacity-50"
>
{colormaps.map((cm) => (
<option key={cm} value={cm}>
{cm.charAt(0).toUpperCase() + cm.slice(1)}
</option>
))}
</select>
</div>
</Card>
);
}

185
frontend/src/components/analysis/HeatmapOverlay.tsx Normal file
View file

@ -0,0 +1,185 @@
import { useEffect, useRef, useState } from "react";
import { useAnalysisStore } from "../../stores/analysisStore";
type ColormapName = "jet" | "viridis" | "inferno" | "magma" | "plasma";
function interpolateColor(
stops: [number, number, number, number][],
t: number,
): [number, number, number, number] {
const n = stops.length - 1;
const idx = Math.min(Math.floor(t * n), n - 1);
const frac = t * n - idx;
const a = stops[idx];
const b = stops[idx + 1];
return [
Math.round(a[0] + (b[0] - a[0]) * frac),
Math.round(a[1] + (b[1] - a[1]) * frac),
Math.round(a[2] + (b[2] - a[2]) * frac),
Math.round(a[3] + (b[3] - a[3]) * frac),
];
}
const colormapStops: Record<ColormapName, [number, number, number, number][]> = {
jet: [
[0, 0, 131, 255],
[0, 60, 170, 255],
[5, 255, 255, 255],
[255, 255, 0, 255],
[250, 0, 0, 255],
[128, 0, 0, 255],
],
viridis: [
[68, 1, 84, 255],
[59, 82, 139, 255],
[33, 145, 140, 255],
[94, 201, 98, 255],
[253, 231, 37, 255],
],
inferno: [
[0, 0, 4, 255],
[40, 11, 84, 255],
[101, 21, 110, 255],
[186, 55, 42, 255],
[252, 163, 4, 255],
[252, 255, 164, 255],
],
magma: [
[0, 0, 4, 255],
[28, 16, 68, 255],
[79, 18, 123, 255],
[182, 54, 121, 255],
[251, 136, 97, 255],
[252, 253, 191, 255],
],
plasma: [
[13, 8, 135, 255],
[84, 2, 163, 255],
[139, 10, 165, 255],
[185, 50, 137, 255],
[219, 92, 104, 255],
[244, 136, 73, 255],
[240, 249, 33, 255],
],
};
function buildColormapLUT(name: ColormapName): Uint8ClampedArray {
const stops = colormapStops[name];
const lut = new Uint8ClampedArray(256 * 4);
for (let i = 0; i < 256; i++) {
const t = i / 255;
const [r, g, b, a] = interpolateColor(stops, t);
lut[i * 4] = r;
lut[i * 4 + 1] = g;
lut[i * 4 + 2] = b;
lut[i * 4 + 3] = a;
}
return lut;
}
interface HeatmapOverlayProps {
originalUrl: string;
saliencyUrl: string;
}
export default function HeatmapOverlay({
originalUrl,
saliencyUrl,
}: HeatmapOverlayProps) {
const canvasRef = useRef<HTMLCanvasElement>(null);
const containerRef = useRef<HTMLDivElement>(null);
const { opacity, colormap, visible } = useAnalysisStore(
(s) => s.heatmapSettings,
);
const zoom = useAnalysisStore((s) => s.zoom);
const [originalImg, setOriginalImg] = useState<HTMLImageElement | null>(null);
const [saliencyImg, setSaliencyImg] = useState<HTMLImageElement | null>(null);
const [canvasSize, setCanvasSize] = useState({ width: 800, height: 600 });
useEffect(() => {
const img = new Image();
img.onload = () => setOriginalImg(img);
img.onerror = (e) => console.error("Failed to load original image:", originalUrl, e);
img.src = originalUrl;
}, [originalUrl]);
useEffect(() => {
const img = new Image();
img.onload = () => setSaliencyImg(img);
img.onerror = (e) => console.error("Failed to load saliency image:", saliencyUrl, e);
img.src = saliencyUrl;
}, [saliencyUrl]);
useEffect(() => {
if (!originalImg) return;
const container = containerRef.current;
if (!container) return;
const maxWidth = container.clientWidth;
const baseScale = Math.min(1, maxWidth / originalImg.naturalWidth);
const scale = baseScale * zoom;
setCanvasSize({
width: Math.round(originalImg.naturalWidth * scale),
height: Math.round(originalImg.naturalHeight * scale),
});
}, [originalImg, zoom]);
useEffect(() => {
const canvas = canvasRef.current;
if (!canvas || !originalImg) return;
const ctx = canvas.getContext("2d");
if (!ctx) return;
canvas.width = canvasSize.width;
canvas.height = canvasSize.height;
ctx.drawImage(originalImg, 0, 0, canvasSize.width, canvasSize.height);
if (visible && saliencyImg) {
const offscreen = document.createElement("canvas");
offscreen.width = canvasSize.width;
offscreen.height = canvasSize.height;
const offCtx = offscreen.getContext("2d")!;
offCtx.drawImage(saliencyImg, 0, 0, canvasSize.width, canvasSize.height);
const saliencyData = offCtx.getImageData(
0,
0,
canvasSize.width,
canvasSize.height,
);
// Find the max gray value to normalize the alpha range
const lut = buildColormapLUT(colormap as ColormapName);
const pixels = saliencyData.data;
let maxGray = 0;
for (let i = 0; i < pixels.length; i += 4) {
if (pixels[i] > maxGray) maxGray = pixels[i];
}
const scale = maxGray > 0 ? 255 / maxGray : 1;
for (let i = 0; i < pixels.length; i += 4) {
const gray = pixels[i];
// Normalize gray to full 0-255 range for better visibility
const normalized = Math.min(255, Math.round(gray * scale));
pixels[i] = lut[normalized * 4];
pixels[i + 1] = lut[normalized * 4 + 1];
pixels[i + 2] = lut[normalized * 4 + 2];
// Use sqrt for a gentler falloff so mid-range values are more visible
pixels[i + 3] = Math.round(255 * opacity * Math.sqrt(normalized / 255));
}
offCtx.putImageData(saliencyData, 0, 0);
ctx.drawImage(offscreen, 0, 0);
}
}, [originalImg, saliencyImg, canvasSize, opacity, colormap, visible]);
return (
<div ref={containerRef} className="w-full">
<canvas
ref={canvasRef}
width={canvasSize.width}
height={canvasSize.height}
className="rounded-lg max-w-full"
/>
</div>
);
}

61
frontend/src/components/analysis/HotspotList.tsx Normal file
View file

@ -0,0 +1,61 @@
import type { Hotspot } from "../../types/analysis";
import Card from "../common/Card";
interface HotspotListProps {
hotspots: Hotspot[];
}
export default function HotspotList({ hotspots }: HotspotListProps) {
const top5 = hotspots.slice(0, 5);
const maxIntensity = top5.length > 0 ? top5[0].intensity : 1;
return (
<Card>
<h3 className="text-sm font-semibold text-gray-700 uppercase tracking-wide mb-4">
Top Attention Hotspots
</h3>
{top5.length === 0 ? (
<p className="text-sm text-gray-400">No hotspots detected.</p>
) : (
<div className="space-y-3">
{top5.map((hotspot) => {
const barWidth = (hotspot.intensity / maxIntensity) * 100;
return (
<div key={hotspot.rank} className="space-y-1">
<div className="flex items-center justify-between text-sm">
<div className="flex items-center gap-2">
<span
className="inline-flex items-center justify-center w-6 h-6 rounded-full text-white text-xs font-bold"
style={{ backgroundColor: "#1a1a2e" }}
>
{hotspot.rank}
</span>
<span className="text-gray-700">
{hotspot.label || `Region ${hotspot.rank}`}
</span>
</div>
<span className="text-gray-500 text-xs">
({Math.round(hotspot.center_x ?? hotspot.x)}, {Math.round(hotspot.center_y ?? hotspot.y)})
</span>
</div>
<div className="w-full bg-gray-100 rounded-full h-3">
<div
className="h-3 rounded-full transition-all duration-500"
style={{
width: `${barWidth}%`,
background: `linear-gradient(90deg, #ffc407, #e0ad06)`,
}}
/>
</div>
<p className="text-xs text-gray-400 text-right">
{(hotspot.intensity * 100).toFixed(1)}% intensity
</p>
</div>
);
})}
</div>
)}
</Card>
);
}

99
frontend/src/components/analysis/InsightsPanel.tsx Normal file
View file

@ -0,0 +1,99 @@
import type { Insight } from "../../types/analysis";
import Card from "../common/Card";
const iconMap = {
success: (
<svg className="w-5 h-5 text-green-500 shrink-0" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M9 12l2 2 4-4m6 2a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
),
info: (
<svg className="w-5 h-5 text-blue-500 shrink-0" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
),
warning: (
<svg className="w-5 h-5 text-amber-500 shrink-0" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M12 9v2m0 4h.01m-6.938 4h13.856c1.54 0 2.502-1.667 1.732-2.5L13.732 4c-.77-.833-1.964-.833-2.732 0L4.082 16.5c-.77.833.192 2.5 1.732 2.5z" />
</svg>
),
};
const borderColorMap = {
success: "border-l-green-500",
info: "border-l-blue-500",
warning: "border-l-amber-500",
};
function InsightCard({ insight, isAI }: { insight: Insight; isAI?: boolean }) {
const bgClass = isAI ? "bg-purple-50/60" : "bg-gray-50";
return (
<div
className={`border-l-4 ${isAI ? "border-l-purple-500" : borderColorMap[insight.type]} ${bgClass} rounded-r-lg p-3 flex gap-3`}
>
{iconMap[insight.type]}
<div>
<p className="text-sm font-semibold text-gray-800">{insight.title}</p>
<p className="text-sm text-gray-600 mt-0.5">{insight.description}</p>
</div>
</div>
);
}
interface InsightsPanelProps {
insights: Insight[];
aiInsights?: Insight[];
aiCostUsd?: number | null;
}
export default function InsightsPanel({ insights, aiInsights, aiCostUsd }: InsightsPanelProps) {
const hasInsights = insights && insights.length > 0;
const hasAI = aiInsights && aiInsights.length > 0;
if (!hasInsights && !hasAI) return null;
return (
<div className="space-y-4">
{hasInsights && (
<Card>
<h3 className="text-sm font-semibold text-gray-700 uppercase tracking-wide mb-3">
Analysis Insights
</h3>
<div className="space-y-2">
{insights.map((insight, i) => (
<InsightCard key={i} insight={insight} />
))}
</div>
</Card>
)}
{hasAI && (
<Card>
<div className="flex items-center gap-2 mb-3">
<h3 className="text-sm font-semibold text-gray-700 uppercase tracking-wide">
AI Design Analysis
</h3>
<span className="text-[10px] font-bold uppercase tracking-wider px-1.5 py-0.5 rounded bg-purple-100 text-purple-700">
AI
</span>
</div>
<div className="space-y-2">
{aiInsights!.map((insight, i) => (
<InsightCard key={i} insight={insight} isAI />
))}
</div>
<div className="flex items-center justify-between mt-3">
<p className="text-[11px] text-gray-400">
Powered by Claude Sonnet 4.6. AI insights are supplementary and should be validated against your design objectives.
</p>
{aiCostUsd != null && (
<span className="text-[11px] text-gray-400 shrink-0 ml-3">
Cost: ${aiCostUsd.toFixed(4)}
</span>
)}
</div>
</Card>
)}
</div>
);
}

334
frontend/src/components/aoi/AOICanvas.tsx Normal file
View file

@ -0,0 +1,334 @@
import { useEffect, useRef, useState, useCallback } from "react";
import { Stage, Layer, Image as KonvaImage, Rect, Transformer, Text, Group } from "react-konva";
import type Konva from "konva";
import { useAnalysisStore } from "../../stores/analysisStore";
import type { AOIRectangle } from "../../types/aoi";
import type { AOIRegion } from "../../types/analysis";
import { createAOIs } from "../../api/analysis";
import Button from "../common/Button";
import Card from "../common/Card";
const AOI_COLORS = ["#ef4444", "#3b82f6", "#f59e0b", "#8b5cf6", "#ec4899", "#ffc407", "#f97316"];
interface AOICanvasProps {
analysisId: string;
imageUrl: string;
}
export default function AOICanvas({ analysisId, imageUrl }: AOICanvasProps) {
const containerRef = useRef<HTMLDivElement>(null);
const transformerRef = useRef<Konva.Transformer>(null);
const [image, setImage] = useState<HTMLImageElement | null>(null);
const [stageSize, setStageSize] = useState({ width: 800, height: 600 });
const [scale, setScale] = useState(1);
const [isDrawing, setIsDrawing] = useState(false);
const [drawMode, setDrawMode] = useState(false);
const [drawStart, setDrawStart] = useState({ x: 0, y: 0 });
const [tempRect, setTempRect] = useState<{ x: number; y: number; width: number; height: number } | null>(null);
const [selectedId, setSelectedId] = useState<string | null>(null);
const [loading, setLoading] = useState(false);
const [editingLabel, setEditingLabel] = useState<string | null>(null);
const regions = useAnalysisStore((s) => s.aoiRegions);
const setRegions = useAnalysisStore((s) => s.setAoiRegions);
const addRegion = useAnalysisStore((s) => s.addAoiRegion);
const updateRegion = useAnalysisStore((s) => s.updateAoiRegion);
const removeRegion = useAnalysisStore((s) => s.removeAoiRegion);
const setAoiResults = useAnalysisStore((s) => s.setAoiResults);
useEffect(() => {
const img = new window.Image();
img.crossOrigin = "anonymous";
img.onload = () => {
setImage(img);
if (containerRef.current) {
const maxWidth = containerRef.current.clientWidth;
const s = Math.min(1, maxWidth / img.naturalWidth);
setScale(s);
setStageSize({
width: Math.round(img.naturalWidth * s),
height: Math.round(img.naturalHeight * s),
});
}
};
img.src = imageUrl;
}, [imageUrl]);
const handleMouseDown = useCallback(
(e: Konva.KonvaEventObject<MouseEvent>) => {
if (!drawMode) return;
const stage = e.target.getStage();
if (!stage) return;
const pos = stage.getPointerPosition();
if (!pos) return;
setIsDrawing(true);
setDrawStart({ x: pos.x, y: pos.y });
setTempRect({ x: pos.x, y: pos.y, width: 0, height: 0 });
},
[drawMode],
);
const handleMouseMove = useCallback(
(e: Konva.KonvaEventObject<MouseEvent>) => {
if (!isDrawing) return;
const stage = e.target.getStage();
if (!stage) return;
const pos = stage.getPointerPosition();
if (!pos) return;
setTempRect({
x: Math.min(drawStart.x, pos.x),
y: Math.min(drawStart.y, pos.y),
width: Math.abs(pos.x - drawStart.x),
height: Math.abs(pos.y - drawStart.y),
});
},
[isDrawing, drawStart],
);
const handleMouseUp = useCallback(() => {
if (!isDrawing || !tempRect) return;
setIsDrawing(false);
if (tempRect.width > 10 && tempRect.height > 10) {
const newRegion: AOIRectangle = {
id: `aoi-${Date.now()}`,
label: `Region ${regions.length + 1}`,
x: tempRect.x / scale,
y: tempRect.y / scale,
width: tempRect.width / scale,
height: tempRect.height / scale,
color: AOI_COLORS[regions.length % AOI_COLORS.length],
};
addRegion(newRegion);
}
setTempRect(null);
setDrawMode(false);
}, [isDrawing, tempRect, regions.length, scale, addRegion]);
const handleDragEnd = useCallback(
(id: string, e: Konva.KonvaEventObject<DragEvent>) => {
updateRegion(id, {
x: e.target.x() / scale,
y: e.target.y() / scale,
});
},
[scale, updateRegion],
);
const handleTransformEnd = useCallback(
(id: string, e: Konva.KonvaEventObject<Event>) => {
const node = e.target;
const scaleX = node.scaleX();
const scaleY = node.scaleY();
node.scaleX(1);
node.scaleY(1);
updateRegion(id, {
x: node.x() / scale,
y: node.y() / scale,
width: (node.width() * scaleX) / scale,
height: (node.height() * scaleY) / scale,
});
},
[scale, updateRegion],
);
useEffect(() => {
if (!transformerRef.current) return;
const stage = transformerRef.current.getStage();
if (!stage || !selectedId) {
transformerRef.current.nodes([]);
return;
}
const node = stage.findOne(`#${selectedId}`);
if (node) {
transformerRef.current.nodes([node]);
} else {
transformerRef.current.nodes([]);
}
transformerRef.current.getLayer()?.batchDraw();
}, [selectedId]);
const handleAnalyze = async () => {
if (regions.length === 0) return;
setLoading(true);
try {
const apiRegions: AOIRegion[] = regions.map((r) => ({
label: r.label,
x: r.x,
y: r.y,
width: r.width,
height: r.height,
}));
const results = await createAOIs(analysisId, apiRegions);
setAoiResults(results);
} catch (err) {
console.error("AOI analysis failed:", err);
} finally {
setLoading(false);
}
};
const handleLabelChange = (id: string, label: string) => {
updateRegion(id, { label });
setEditingLabel(null);
};
return (
<div className="space-y-4">
<div className="flex items-center gap-3 flex-wrap">
<Button
variant={drawMode ? "secondary" : "primary"}
onClick={() => setDrawMode(!drawMode)}
>
{drawMode ? "Cancel Drawing" : "Add Region"}
</Button>
<Button
variant="primary"
onClick={handleAnalyze}
loading={loading}
disabled={regions.length === 0}
>
Analyze AOIs
</Button>
{regions.length > 0 && (
<Button
variant="danger"
onClick={() => setRegions([])}
>
Clear All
</Button>
)}
{drawMode && (
<span className="text-sm text-gray-500">
Click and drag on the image to draw a region
</span>
)}
</div>
<div
ref={containerRef}
className="border border-gray-200 rounded-lg overflow-hidden"
style={{ cursor: drawMode ? "crosshair" : "default" }}
>
{image && (
<Stage
width={stageSize.width}
height={stageSize.height}
onMouseDown={handleMouseDown}
onMouseMove={handleMouseMove}
onMouseUp={handleMouseUp}
onClick={(e) => {
if (e.target === e.target.getStage()) {
setSelectedId(null);
}
}}
>
<Layer>
<KonvaImage
image={image}
width={stageSize.width}
height={stageSize.height}
/>
{regions.map((region) => (
<Group key={region.id}>
<Rect
id={region.id}
x={region.x * scale}
y={region.y * scale}
width={region.width * scale}
height={region.height * scale}
stroke={region.color}
strokeWidth={2}
fill={`${region.color}22`}
draggable={!drawMode}
onClick={() => setSelectedId(region.id)}
onDragEnd={(e) => handleDragEnd(region.id, e)}
onTransformEnd={(e) => handleTransformEnd(region.id, e)}
/>
<Text
x={region.x * scale + 4}
y={region.y * scale + 4}
text={region.label}
fontSize={12}
fill="white"
padding={2}
/>
</Group>
))}
{tempRect && (
<Rect
x={tempRect.x}
y={tempRect.y}
width={tempRect.width}
height={tempRect.height}
stroke="#ffc407"
strokeWidth={2}
fill="#ffc40722"
dash={[4, 4]}
/>
)}
<Transformer
ref={transformerRef}
rotateEnabled={false}
keepRatio={false}
boundBoxFunc={(oldBox, newBox) => {
if (newBox.width < 10 || newBox.height < 10) return oldBox;
return newBox;
}}
/>
</Layer>
</Stage>
)}
</div>
{regions.length > 0 && (
<Card>
<h4 className="text-sm font-semibold text-gray-700 mb-2">
Defined Regions
</h4>
<div className="space-y-2">
{regions.map((r) => (
<div
key={r.id}
className="flex items-center gap-3 text-sm py-1"
>
<span
className="w-3 h-3 rounded-sm flex-shrink-0"
style={{ backgroundColor: r.color }}
/>
{editingLabel === r.id ? (
<input
autoFocus
defaultValue={r.label}
onBlur={(e) => handleLabelChange(r.id, e.target.value)}
onKeyDown={(e) => {
if (e.key === "Enter") {
handleLabelChange(r.id, e.currentTarget.value);
}
}}
className="border border-gray-300 rounded px-2 py-0.5 text-sm flex-1"
/>
) : (
<span
className="text-gray-700 cursor-pointer hover:text-[#ffc407] flex-1"
onClick={() => setEditingLabel(r.id)}
>
{r.label}
</span>
)}
<span className="text-gray-400 text-xs">
{Math.round(r.width)}x{Math.round(r.height)}
</span>
<button
onClick={() => removeRegion(r.id)}
className="text-red-400 hover:text-red-600 text-xs"
>
Remove
</button>
</div>
))}
</div>
</Card>
)}
</div>
);
}

91
frontend/src/components/aoi/AOIResults.tsx Normal file
View file

@ -0,0 +1,91 @@
import type { AOIResult } from "../../types/analysis";
import Card from "../common/Card";
interface AOIResultsProps {
results: AOIResult[];
}
export default function AOIResults({ results }: AOIResultsProps) {
if (results.length === 0) return null;
return (
<Card>
<h3 className="text-sm font-semibold text-gray-700 uppercase tracking-wide mb-4">
AOI Analysis Results
</h3>
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead>
<tr className="border-b border-gray-200">
<th className="text-left py-2 px-3 text-gray-500 font-medium">
Region
</th>
<th className="text-right py-2 px-3 text-gray-500 font-medium">
Attention %
</th>
<th className="text-right py-2 px-3 text-gray-500 font-medium">
Area %
</th>
<th className="text-right py-2 px-3 text-gray-500 font-medium">
Density
</th>
</tr>
</thead>
<tbody>
{results.map((result, idx) => {
const densityColor =
result.density >= 1 ? "#ffc407" : "#ef4444";
return (
<tr
key={result.id || idx}
className="border-b border-gray-100 hover:bg-gray-50"
>
<td className="py-2.5 px-3 font-medium text-gray-700">
{result.label}
</td>
<td className="py-2.5 px-3 text-right">
<div className="flex items-center justify-end gap-2">
<div className="w-16 bg-gray-200 rounded-full h-2">
<div
className="h-2 rounded-full"
style={{
width: `${Math.min(result.attention_percent, 100)}%`,
backgroundColor: "#ffc407",
}}
/>
</div>
<span className="text-gray-700 w-14 text-right">
{result.attention_percent.toFixed(1)}%
</span>
</div>
</td>
<td className="py-2.5 px-3 text-right text-gray-700">
{result.area_percent.toFixed(1)}%
</td>
<td className="py-2.5 px-3 text-right">
<span
className="inline-block px-2 py-0.5 rounded text-xs font-semibold"
style={{
color: densityColor,
backgroundColor:
result.density >= 1
? "#ffc40720"
: "#ef444420",
}}
>
{result.density.toFixed(2)}x
</span>
</td>
</tr>
);
})}
</tbody>
</table>
</div>
<p className="text-xs text-gray-400 mt-3">
Density = Attention% / Area%. Values above 1.0 indicate the region
attracts more attention than its size would suggest.
</p>
</Card>
);
}

64
frontend/src/components/common/Button.tsx Normal file
View file

@ -0,0 +1,64 @@
import type { ButtonHTMLAttributes, ReactNode } from "react";
type Variant = "primary" | "secondary" | "danger";
interface ButtonProps extends ButtonHTMLAttributes<HTMLButtonElement> {
variant?: Variant;
children: ReactNode;
loading?: boolean;
}
const variantClasses: Record<Variant, string> = {
primary: "text-gray-900",
secondary:
"bg-transparent border-2 border-gray-300 text-gray-700 hover:bg-gray-50",
danger: "bg-red-500 text-white hover:bg-red-600",
};
const variantStyles: Record<Variant, React.CSSProperties> = {
primary: { backgroundColor: "#ffc407" },
secondary: {},
danger: {},
};
export default function Button({
variant = "primary",
children,
loading = false,
className = "",
disabled,
style,
...props
}: ButtonProps) {
return (
<button
className={`inline-flex items-center justify-center gap-2 px-4 py-2 rounded-lg text-sm font-medium transition-all duration-200 disabled:opacity-50 disabled:cursor-not-allowed ${variantClasses[variant]} ${className}`}
style={{ ...variantStyles[variant], ...style }}
disabled={disabled || loading}
{...props}
>
{loading && (
<svg
className="animate-spin w-4 h-4"
viewBox="0 0 24 24"
fill="none"
>
<circle
className="opacity-25"
cx="12"
cy="12"
r="10"
stroke="currentColor"
strokeWidth="4"
/>
<path
className="opacity-75"
fill="currentColor"
d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4z"
/>
</svg>
)}
{children}
</button>
);
}

23
frontend/src/components/common/Card.tsx Normal file
View file

@ -0,0 +1,23 @@
import type { ReactNode } from "react";
interface CardProps {
children: ReactNode;
className?: string;
padding?: boolean;
}
export default function Card({
children,
className = "",
padding = true,
}: CardProps) {
return (
<div
className={`bg-white rounded-xl shadow-sm border border-gray-200 ${
padding ? "p-6" : ""
} ${className}`}
>
{children}
</div>
);
}

40
frontend/src/components/common/LoadingSpinner.tsx Normal file
View file

@ -0,0 +1,40 @@
interface LoadingSpinnerProps {
size?: "sm" | "md" | "lg";
message?: string;
}
const sizeClasses = {
sm: "w-5 h-5",
md: "w-8 h-8",
lg: "w-12 h-12",
};
export default function LoadingSpinner({
size = "md",
message,
}: LoadingSpinnerProps) {
return (
<div className="flex flex-col items-center justify-center gap-3 py-8">
<svg
className={`animate-spin ${sizeClasses[size]}`}
viewBox="0 0 24 24"
fill="none"
>
<circle
className="opacity-25"
cx="12"
cy="12"
r="10"
stroke="#ffc407"
strokeWidth="4"
/>
<path
className="opacity-75"
fill="#ffc407"
d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4z"
/>
</svg>
{message && <p className="text-sm text-gray-500">{message}</p>}
</div>
);
}

24
frontend/src/components/layout/AppLayout.tsx Normal file
View file

@ -0,0 +1,24 @@
import { Outlet } from "react-router-dom";
import Header from "./Header";
import Sidebar from "./Sidebar";
import { useUIStore } from "../../stores/uiStore";
export default function AppLayout() {
const sidebarOpen = useUIStore((s) => s.sidebarOpen);
return (
<div className="min-h-screen bg-gray-50">
<Header />
<Sidebar />
<main
className={`pt-16 transition-all duration-300 ${
sidebarOpen ? "ml-56" : "ml-16"
}`}
>
<div className="p-6">
<Outlet />
</div>
</main>
</div>
);
}

77
frontend/src/components/layout/Header.tsx Normal file
View file

@ -0,0 +1,77 @@
import { Link } from "react-router-dom";
import { useUIStore } from "../../stores/uiStore";
export default function Header() {
const toggleSidebar = useUIStore((s) => s.toggleSidebar);
return (
<header
className="fixed top-0 left-0 right-0 z-50 flex items-center justify-between px-6 h-16 shadow-sm"
style={{ backgroundColor: "#1a1a2e" }}
>
<div className="flex items-center gap-4">
<button
onClick={toggleSidebar}
className="text-white/70 hover:text-white p-1 rounded transition-colors"
aria-label="Toggle sidebar"
>
<svg
className="w-6 h-6"
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
strokeWidth={2}
d="M4 6h16M4 12h16M4 18h16"
/>
</svg>
</button>
<Link to="/" className="flex items-center gap-2">
<span className="text-2xl font-bold text-white tracking-tight">
Oli<span style={{ color: "#ffc407" }}>VAS</span>
</span>
<span className="text-xs text-white/50 hidden sm:inline mt-1">
Visual Attention Suite
</span>
</Link>
</div>
<nav className="flex items-center gap-3">
<Link
to="/"
className="text-sm text-white/70 hover:text-white transition-colors px-3 py-1.5 rounded"
>
Dashboard
</Link>
<Link
to="/help"
className="text-sm text-white/70 hover:text-white transition-colors px-3 py-1.5 rounded"
>
Help
</Link>
<Link
to="/about"
className="text-sm text-white/70 hover:text-white transition-colors px-3 py-1.5 rounded"
>
About
</Link>
<Link
to="/analyze"
className="text-sm font-medium text-white px-4 py-1.5 rounded transition-colors"
style={{ backgroundColor: "#ffc407", color: "#1a1a2e" }}
onMouseEnter={(e) =>
(e.currentTarget.style.backgroundColor = "#e0ad06")
}
onMouseLeave={(e) =>
(e.currentTarget.style.backgroundColor = "#ffc407")
}
>
New Analysis
</Link>
</nav>
</header>
);
}

80
frontend/src/components/layout/Sidebar.tsx Normal file
View file

@ -0,0 +1,80 @@
import { NavLink } from "react-router-dom";
import { useUIStore } from "../../stores/uiStore";
const navItems = [
{
to: "/",
label: "Dashboard",
icon: (
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2}
d="M3 12l2-2m0 0l7-7 7 7M5 10v10a1 1 0 001 1h3m10-11l2 2m-2-2v10a1 1 0 01-1 1h-3m-4 0a1 1 0 01-1-1v-4a1 1 0 011-1h2a1 1 0 011 1v4a1 1 0 01-1 1" />
</svg>
),
},
{
to: "/analyze",
label: "New Analysis",
icon: (
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2}
d="M12 4v16m8-8H4" />
</svg>
),
},
{
to: "/help",
label: "Help",
icon: (
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2}
d="M8.228 9c.549-1.165 2.03-2 3.772-2 2.21 0 4 1.343 4 3 0 1.4-1.278 2.575-3.006 2.907-.542.104-.994.54-.994 1.093m0 3h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
),
},
{
to: "/about",
label: "About",
icon: (
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2}
d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
),
},
];
export default function Sidebar() {
const sidebarOpen = useUIStore((s) => s.sidebarOpen);
return (
<aside
className={`fixed left-0 top-16 bottom-0 z-40 bg-white border-r border-gray-200 transition-all duration-300 ${
sidebarOpen ? "w-56" : "w-16"
}`}
>
<nav className="flex flex-col gap-1 p-3 mt-2">
{navItems.map((item) => (
<NavLink
key={item.to}
to={item.to}
end={item.to === "/"}
className={({ isActive }) =>
`flex items-center gap-3 px-3 py-2.5 rounded-lg text-sm font-medium transition-colors ${
isActive
? "text-gray-900"
: "text-gray-600 hover:bg-gray-100 hover:text-gray-900"
}`
}
style={({ isActive }) =>
isActive ? { backgroundColor: "#ffc407", color: "#1a1a2e" } : {}
}
>
{item.icon}
{sidebarOpen && <span>{item.label}</span>}
</NavLink>
))}
</nav>
</aside>
);
}

96
frontend/src/components/upload/DropZone.tsx Normal file
View file

@ -0,0 +1,96 @@
import { useCallback, useState } from "react";
import { useDropzone } from "react-dropzone";
interface DropZoneProps {
onFileSelected: (file: File) => void;
disabled?: boolean;
}
export default function DropZone({ onFileSelected, disabled }: DropZoneProps) {
const [preview, setPreview] = useState<string | null>(null);
const [selectedFile, setSelectedFile] = useState<File | null>(null);
const onDrop = useCallback(
(acceptedFiles: File[]) => {
const file = acceptedFiles[0];
if (!file) return;
setSelectedFile(file);
setPreview(URL.createObjectURL(file));
onFileSelected(file);
},
[onFileSelected],
);
const { getRootProps, getInputProps, isDragActive } = useDropzone({
onDrop,
accept: {
"image/jpeg": [".jpg", ".jpeg"],
"image/png": [".png"],
"image/tiff": [".tiff", ".tif"],
"image/webp": [".webp"],
},
maxFiles: 1,
disabled,
});
return (
<div className="space-y-4">
<div
{...getRootProps()}
className={`relative border-2 border-dashed rounded-xl p-8 text-center cursor-pointer transition-all duration-200 ${
isDragActive
? "border-[#ffc407] bg-[#ffc407]/5"
: "border-gray-300 hover:border-[#ffc407] hover:bg-gray-50"
} ${disabled ? "opacity-50 cursor-not-allowed" : ""}`}
>
<input {...getInputProps()} />
{preview ? (
<div className="flex flex-col items-center gap-4">
<img
src={preview}
alt="Preview"
className="max-h-48 rounded-lg object-contain"
/>
<div className="text-sm text-gray-600">
<p className="font-medium">{selectedFile?.name}</p>
<p className="text-gray-400 mt-1">
{selectedFile
? `${(selectedFile.size / 1024 / 1024).toFixed(2)} MB`
: ""}
</p>
</div>
<p className="text-xs text-gray-400">
Drop a new image to replace
</p>
</div>
) : (
<div className="flex flex-col items-center gap-3">
<svg
className="w-12 h-12 text-gray-400"
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
strokeWidth={1.5}
d="M4 16l4.586-4.586a2 2 0 012.828 0L16 16m-2-2l1.586-1.586a2 2 0 012.828 0L20 14m-6-6h.01M6 20h12a2 2 0 002-2V6a2 2 0 00-2-2H6a2 2 0 00-2 2v12a2 2 0 002 2z"
/>
</svg>
<div>
<p className="text-sm font-medium text-gray-700">
{isDragActive
? "Drop your image here"
: "Drag and drop an image, or click to browse"}
</p>
<p className="text-xs text-gray-400 mt-1">
JPEG, PNG, TIFF, or WebP
</p>
</div>
</div>
)}
</div>
</div>
);
}

51
frontend/src/components/upload/UploadPreview.tsx Normal file
View file

@ -0,0 +1,51 @@
import { useEffect, useState } from "react";
import Card from "../common/Card";
interface UploadPreviewProps {
file: File;
}
export default function UploadPreview({ file }: UploadPreviewProps) {
const [dimensions, setDimensions] = useState<{
width: number;
height: number;
} | null>(null);
const [previewUrl, setPreviewUrl] = useState<string>("");
useEffect(() => {
const url = URL.createObjectURL(file);
setPreviewUrl(url);
const img = new Image();
img.onload = () => {
setDimensions({ width: img.naturalWidth, height: img.naturalHeight });
};
img.src = url;
return () => URL.revokeObjectURL(url);
}, [file]);
const sizeMB = (file.size / 1024 / 1024).toFixed(2);
return (
<Card className="flex gap-4 items-start">
{previewUrl && (
<img
src={previewUrl}
alt="Preview"
className="w-24 h-24 rounded-lg object-cover flex-shrink-0"
/>
)}
<div className="text-sm space-y-1">
<p className="font-medium text-gray-800">{file.name}</p>
<p className="text-gray-500">{sizeMB} MB</p>
{dimensions && (
<p className="text-gray-500">
{dimensions.width} x {dimensions.height} px
</p>
)}
<p className="text-gray-400">{file.type}</p>
</div>
</Card>
);
}

13
frontend/src/globals.css Normal file
View file

@ -0,0 +1,13 @@
@import "tailwindcss";
body {
font-family:
"Montserrat",
-apple-system,
BlinkMacSystemFont,
"Segoe UI",
Roboto,
sans-serif;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
}

27
frontend/src/hooks/useAnalysis.ts Normal file
View file

@ -0,0 +1,27 @@
import { useQuery } from "@tanstack/react-query";
import { getAnalysis, getAnalysisStatus } from "../api/analysis";
import type { AnalysisDetail, AnalysisStatus } from "../types/analysis";
export function useAnalysis(analysisId: string | undefined) {
return useQuery<AnalysisDetail>({
queryKey: ["analysis", analysisId],
queryFn: () => getAnalysis(analysisId!),
enabled: !!analysisId,
});
}
export function useAnalysisStatus(
analysisId: string | undefined,
enabled: boolean = true,
) {
return useQuery<AnalysisStatus>({
queryKey: ["analysisStatus", analysisId],
queryFn: () => getAnalysisStatus(analysisId!),
enabled: !!analysisId && enabled,
refetchInterval: (query) => {
const status = query.state.data?.status;
if (status === "completed" || status === "failed") return false;
return 2000;
},
});
}

63
frontend/src/hooks/useProjects.ts Normal file
View file

@ -0,0 +1,63 @@
import { useQuery, useMutation, useQueryClient } from "@tanstack/react-query";
import {
listProjects,
getProject,
createProject,
updateProject,
deleteProject,
} from "../api/projects";
import type { Project, ProjectDetail, ProjectCreate } from "../types/project";
export function useProjects(page: number = 1, perPage: number = 20) {
return useQuery<Project[]>({
queryKey: ["projects", page, perPage],
queryFn: () => listProjects(page, perPage),
});
}
export function useProject(projectId: string | undefined) {
return useQuery<ProjectDetail>({
queryKey: ["project", projectId],
queryFn: () => getProject(projectId!),
enabled: !!projectId,
});
}
export function useCreateProject() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ name, description }: ProjectCreate) =>
createProject(name, description),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ["projects"] });
},
});
}
export function useUpdateProject() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({
id,
...updates
}: Partial<ProjectCreate> & { id: string }) =>
updateProject(id, updates),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ["projects"] });
queryClient.invalidateQueries({ queryKey: ["project", variables.id] });
},
});
}
export function useDeleteProject() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (id: string) => deleteProject(id),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ["projects"] });
},
});
}

48
frontend/src/hooks/useUpload.ts Normal file
View file

@ -0,0 +1,48 @@
import { useState, useCallback } from "react";
import { useMutation } from "@tanstack/react-query";
import { createAnalysis } from "../api/analysis";
import type { AnalysisDetail } from "../types/analysis";
interface UseUploadOptions {
onSuccess?: (analysis: AnalysisDetail) => void;
onError?: (error: Error) => void;
}
export function useUpload(options?: UseUploadOptions) {
const [progress, setProgress] = useState(0);
const mutation = useMutation<
AnalysisDetail,
Error,
{ projectId: string; file: File; name?: string; model?: string }
>({
mutationFn: async ({ projectId, file, name, model }) => {
setProgress(10);
const result = await createAnalysis(projectId, file, name, model);
setProgress(100);
return result;
},
onSuccess: (data) => {
options?.onSuccess?.(data);
},
onError: (error) => {
setProgress(0);
options?.onError?.(error);
},
});
const reset = useCallback(() => {
setProgress(0);
mutation.reset();
}, [mutation]);
return {
upload: mutation.mutate,
uploadAsync: mutation.mutateAsync,
isUploading: mutation.isPending,
progress,
error: mutation.error,
data: mutation.data,
reset,
};
}

26
frontend/src/main.tsx Normal file
View file

@ -0,0 +1,26 @@
import { StrictMode } from "react";
import { createRoot } from "react-dom/client";
import { BrowserRouter } from "react-router-dom";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import App from "./App";
import "./globals.css";
const queryClient = new QueryClient({
defaultOptions: {
queries: {
retry: 1,
refetchOnWindowFocus: false,
staleTime: 30_000,
},
},
});
createRoot(document.getElementById("root")!).render(
<StrictMode>
<QueryClientProvider client={queryClient}>
<BrowserRouter>
<App />
</BrowserRouter>
</QueryClientProvider>
</StrictMode>,
);

468
frontend/src/pages/About.tsx Normal file
View file

@ -0,0 +1,468 @@
import { useState } from "react";
import Card from "../components/common/Card";
function Section({
title,
children,
}: {
title: string;
children: React.ReactNode;
}) {
return (
<Card>
<h2 className="text-lg font-semibold mb-3" style={{ color: "#1a1a2e" }}>
{title}
</h2>
<div className="text-sm text-gray-600 space-y-3 leading-relaxed">
{children}
</div>
</Card>
);
}
function Expandable({
label,
children,
}: {
label: string;
children: React.ReactNode;
}) {
const [open, setOpen] = useState(false);
return (
<div className="mt-2">
<button
onClick={() => setOpen(!open)}
className="text-sm font-medium text-blue-600 hover:text-blue-800 flex items-center gap-1"
>
<svg
className={`w-4 h-4 transition-transform ${open ? "rotate-90" : ""}`}
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
strokeWidth={2}
d="M9 5l7 7-7 7"
/>
</svg>
{label}
</button>
{open && (
<div className="mt-2 pl-5 text-xs text-gray-500 space-y-2 border-l-2 border-gray-200">
{children}
</div>
)}
</div>
);
}
function Citation({
authors,
year,
title,
venue,
url,
}: {
authors: string;
year: string;
title: string;
venue: string;
url: string;
}) {
return (
<p className="text-xs text-gray-500">
{authors} ({year}).{" "}
<a
href={url}
target="_blank"
rel="noopener noreferrer"
className="text-blue-600 hover:underline"
>
&ldquo;{title}&rdquo;
</a>
. <em>{venue}</em>.
</p>
);
}
export default function About() {
return (
<div className="max-w-3xl mx-auto space-y-6">
{/* Hero */}
<div>
<h1 className="text-2xl font-bold" style={{ color: "#1a1a2e" }}>
The Science Behind Oli
<span style={{ color: "#ffc407" }}>VAS</span>
</h1>
<p className="text-gray-500 text-sm mt-1">
How we predict where people look — and why you can trust the results
</p>
</div>
{/* Layman's explanation */}
<Section title="How It Works">
<p>
When you look at an image — an ad, a product package, a web page — your
eyes don't scan everything equally. In the first 3–5 seconds, your gaze
is drawn to specific areas: faces, high-contrast text, bright colours,
and unusual shapes. This is called <strong>visual saliency</strong>.
</p>
<p>
OliVAS uses <strong>deep neural networks</strong> to predict this
behaviour. These networks were trained on data from thousands of{" "}
<strong>real eye-tracking experiments</strong>, where researchers recorded
exactly where people looked when shown different images. The models
learned the patterns that drive human attention — and can now predict
them for any new image you upload.
</p>
<p>
The result is a <strong>heatmap</strong> showing where viewers are most
likely to look, a <strong>gaze sequence</strong> predicting the order of
fixations, and <strong>attention metrics</strong> that quantify how
effectively your design captures attention.
</p>
</Section>
{/* The Models */}
<Section title="Our Saliency Models">
<p>
OliVAS uses the <strong>DeepGaze</strong> family of models, developed at
the University of Tübingen by Matthias Kümmerer, Matthias Bethge, and
colleagues. These are consistently among the{" "}
<strong>top-ranked models</strong> on the{" "}
<a
href="https://saliency.tuebingen.ai/"
target="_blank"
rel="noopener noreferrer"
className="text-blue-600 hover:underline"
>
MIT/Tübingen Saliency Benchmark
</a>
, the standard evaluation platform for saliency prediction.
</p>
{/* DeepGaze IIE */}
<div className="bg-gray-50 rounded-lg p-4 border border-gray-200">
<div className="flex items-center gap-2 mb-2">
<h3 className="font-semibold text-gray-800">DeepGaze IIE</h3>
<span
className="text-xs px-2 py-0.5 rounded-full font-medium"
style={{ backgroundColor: "#ffc407", color: "#1a1a2e" }}
>
Recommended
</span>
</div>
<p>
Our recommended model for most use cases. It combines features from
two powerful image recognition networks (ResNet and DenseNet) to
predict where people look. It is <strong>calibrated</strong> — meaning
its confidence scores closely match actual fixation probabilities, even
on images very different from its training data.
</p>
<Expandable label="Technical Details">
<p>
DeepGaze IIE uses an ensemble of ResNet-50 and DenseNet-201 features
with a readout network and learned center bias prior. It was the
first model to demonstrate strong out-of-domain generalisation across
different saliency datasets. It achieves state-of-the-art performance
on the MIT/Tübingen benchmark with an Information Gain (IG) score
that significantly outperforms previous models.
</p>
<Citation
authors="Linardos, A., Kümmerer, M., Press, O., & Bethge, M."
year="2021"
title="DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling"
venue="Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 12919–12928"
url="https://arxiv.org/abs/2105.12441"
/>
</Expandable>
</div>
{/* DeepGaze III */}
<div className="bg-gray-50 rounded-lg p-4 border border-gray-200">
<h3 className="font-semibold text-gray-800 mb-2">DeepGaze III</h3>
<p>
The latest model in the family, designed to predict not just{" "}
<em>where</em> people look but the <em>sequence</em> of fixations
over time. It uses a transformer-based architecture that can capture
long-range dependencies in an image — useful for complex layouts with
many elements.
</p>
<Expandable label="Technical Details">
<p>
DeepGaze III models scanpaths (sequences of fixations) rather than
just static saliency maps. It uses a deep neural network with a
fixation selection mechanism that accounts for the temporal dynamics
of visual exploration. Published in the Journal of Vision, the
leading peer-reviewed journal for vision science.
</p>
<Citation
authors="Kümmerer, M., Bethge, M., & Wallis, T.S.A."
year="2022"
title="DeepGaze III: Modeling free-viewing human scanpaths with deep learning"
venue="Journal of Vision, 22(5):7"
url="https://doi.org/10.1167/jov.22.5.7"
/>
</Expandable>
</div>
{/* DeepGaze I */}
<div className="bg-gray-50 rounded-lg p-4 border border-gray-200">
<h3 className="font-semibold text-gray-800 mb-2">DeepGaze I</h3>
<p>
The original DeepGaze model that pioneered using deep learning features
for saliency prediction. It demonstrated that features learned for
object recognition (on ImageNet) naturally encode information about
where humans look. Faster than the newer variants but less accurate on
complex designs.
</p>
<Expandable label="Technical Details">
<p>
DeepGaze I uses features from AlexNet (trained on ImageNet) with a
linear readout to predict fixation density. It was the first model to
show that deep features trained for object classification transfer
well to saliency prediction, significantly outperforming
hand-engineered feature models.
</p>
<Citation
authors="Kümmerer, M., Theis, L., & Bethge, M."
year="2015"
title="Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet"
venue="ICLR 2015 Workshop"
url="https://arxiv.org/abs/1411.1045"
/>
</Expandable>
</div>
</Section>
{/* Validation */}
<Section title="Validation and Benchmarks">
<p>
How do we know these predictions are accurate? The models are evaluated
against <strong>real human eye-tracking data</strong>. In eye-tracking
studies, participants view images while a camera tracks their exact gaze
position at high speed (typically 250–1000 times per second). This
produces ground truth "fixation maps" showing where humans actually
looked.
</p>
<p>
The{" "}
<a
href="https://saliency.tuebingen.ai/"
target="_blank"
rel="noopener noreferrer"
className="text-blue-600 hover:underline"
>
MIT/Tübingen Saliency Benchmark
</a>{" "}
is the standard platform for comparing saliency models. It uses hidden
test sets so models cannot be tuned to the evaluation data. DeepGaze
models consistently rank among the top performers.
</p>
<p>
Key evaluation datasets include:
</p>
<ul className="list-disc pl-5 space-y-1">
<li>
<strong>MIT1003</strong> — 1,003 natural images with eye-tracking data
from 15 viewers each (
<a
href="https://doi.org/10.1109/ICCV.2009.5459462"
target="_blank"
rel="noopener noreferrer"
className="text-blue-600 hover:underline"
>
Judd et al., 2009
</a>
)
</li>
<li>
<strong>CAT2000</strong> — 4,000 images across 20 categories with
fixation data from 24 observers (
<a
href="https://arxiv.org/abs/1505.03581"
target="_blank"
rel="noopener noreferrer"
className="text-blue-600 hover:underline"
>
Borji & Itti, 2015
</a>
)
</li>
</ul>
<Expandable label="About the benchmark methodology">
<p>
Models are evaluated on multiple metrics: AUC (Area Under the ROC
Curve), Information Gain (IG), Normalized Scanpath Saliency (NSS),
and KL Divergence. These measure different aspects of prediction
quality — from binary "is this a fixated location" to fine-grained
probability calibration.
</p>
<p>
DeepGaze IIE achieves some of the highest Information Gain scores on
the MIT benchmark, meaning its predicted probability distributions
closely match actual human fixation patterns.
</p>
</Expandable>
</Section>
{/* Foundational Research */}
<Section title="Historical Context">
<p>
The scientific study of visual attention goes back decades. In 1998,
Laurent Itti, Christof Koch, and Ernst Niebur published a landmark
computational model of visual attention based on neuroscience principles
— centre-surround contrast in colour, intensity, and orientation. This
model established the field of computational saliency prediction.
</p>
<p>
Since then, the field has evolved from hand-crafted feature models to
deep learning approaches. DeepGaze I (2015) was a pivotal moment,
showing that neural networks trained for object recognition naturally
learn to predict human attention. Each subsequent version has improved
accuracy and generalisation.
</p>
<Expandable label="Foundational papers">
<Citation
authors="Itti, L., Koch, C., & Niebur, E."
year="1998"
title="A Model of Saliency-Based Visual Attention for Rapid Scene Analysis"
venue="IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254–1259"
url="https://doi.org/10.1109/34.730558"
/>
<Citation
authors="Kümmerer, M., Wallis, T.S.A., & Bethge, M."
year="2016"
title="DeepGaze II: Reading fixations from deep features trained on object recognition"
venue="arXiv preprint"
url="https://arxiv.org/abs/1610.01563"
/>
</Expandable>
</Section>
{/* Important caveats */}
<Section title="Limitations and Caveats">
<p>
While saliency models are powerful tools, they have important
limitations to keep in mind:
</p>
<ul className="list-disc pl-5 space-y-2">
<li>
<strong>First-glance only</strong> — Predictions model the first 3–5
seconds of free viewing. They do not predict reading behaviour,
task-driven search, or extended engagement.
</li>
<li>
<strong>Bottom-up attention</strong> — Models predict attention driven
by visual features (contrast, faces, objects). They do not account for
top-down factors like user intent, cultural context, or brand
recognition.
</li>
<li>
<strong>Aggregated viewers</strong> — Predictions represent an
average across many viewers. Individual variation can be substantial.
</li>
<li>
<strong>Static images only</strong> — Models are trained on and
predict for static images, not video or interactive content.
</li>
</ul>
<p>
For best results, use OliVAS as one input in your design review process
alongside user testing, A/B testing, and domain expertise.
</p>
</Section>
{/* Full References */}
<Section title="References">
<div className="space-y-3">
<Citation
authors="Borji, A. & Itti, L."
year="2015"
title="CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research"
venue="CVPR 2015 Workshop on Future of Datasets"
url="https://arxiv.org/abs/1505.03581"
/>
<Citation
authors="Itti, L., Koch, C., & Niebur, E."
year="1998"
title="A Model of Saliency-Based Visual Attention for Rapid Scene Analysis"
venue="IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254–1259"
url="https://doi.org/10.1109/34.730558"
/>
<Citation
authors="Judd, T., Ehinger, K., Durand, F., & Torralba, A."
year="2009"
title="Learning to Predict Where Humans Look"
venue="IEEE International Conference on Computer Vision (ICCV), pp. 2106–2113"
url="https://doi.org/10.1109/ICCV.2009.5459462"
/>
<Citation
authors="Kümmerer, M., Theis, L., & Bethge, M."
year="2015"
title="Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet"
venue="ICLR 2015 Workshop"
url="https://arxiv.org/abs/1411.1045"
/>
<Citation
authors="Kümmerer, M., Wallis, T.S.A., & Bethge, M."
year="2016"
title="DeepGaze II: Reading fixations from deep features trained on object recognition"
venue="arXiv preprint"
url="https://arxiv.org/abs/1610.01563"
/>
<Citation
authors="Linardos, A., Kümmerer, M., Press, O., & Bethge, M."
year="2021"
title="DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling"
venue="IEEE/CVF International Conference on Computer Vision (ICCV), pp. 12919–12928"
url="https://arxiv.org/abs/2105.12441"
/>
<Citation
authors="Kümmerer, M., Bethge, M., & Wallis, T.S.A."
year="2022"
title="DeepGaze III: Modeling free-viewing human scanpaths with deep learning"
venue="Journal of Vision, 22(5):7"
url="https://doi.org/10.1167/jov.22.5.7"
/>
<Citation
authors="Radford, A., Kim, J.W., Hallacy, C., et al."
year="2021"
title="Learning Transferable Visual Models From Natural Language Supervision"
venue="Proceedings of the 38th International Conference on Machine Learning (ICML), PMLR 139:8748–8763"
url="https://arxiv.org/abs/2103.00020"
/>
</div>
<div className="mt-4 pt-3 border-t border-gray-200">
<p className="text-xs text-gray-400">
Open-source implementation:{" "}
<a
href="https://github.com/matthias-k/DeepGaze"
target="_blank"
rel="noopener noreferrer"
className="text-blue-600 hover:underline"
>
github.com/matthias-k/DeepGaze
</a>{" "}
| Benchmark:{" "}
<a
href="https://saliency.tuebingen.ai/"
target="_blank"
rel="noopener noreferrer"
className="text-blue-600 hover:underline"
>
saliency.tuebingen.ai
</a>
</p>
</div>
</Section>
<div className="text-center py-6 text-xs text-gray-400">
OliVAS — Open-Source Visual Attention Software by OLIVER
</div>
</div>
);
}

274
frontend/src/pages/AnalysisView.tsx Normal file
View file

@ -0,0 +1,274 @@
import { useEffect, useState } from "react";
import { useParams } from "react-router-dom";
import { useAnalysis } from "../hooks/useAnalysis";
import { useAnalysisStore, type AnalysisTab } from "../stores/analysisStore";
import { getAnalysisImageUrl, checkAIInsightsAvailable, generateAIInsights } from "../api/analysis";
import type { Insight } from "../types/analysis";
function ZoomableImage({ src, alt }: { src: string; alt: string }) {
const zoom = useAnalysisStore((s) => s.zoom);
return (
<div className="overflow-auto">
<img
src={src}
alt={alt}
className="object-contain"
style={{ width: `${zoom * 100}%`, maxWidth: "none" }}
/>
</div>
);
}
function ZoomControls() {
const zoom = useAnalysisStore((s) => s.zoom);
const setZoom = useAnalysisStore((s) => s.setZoom);
return (
<div className="flex items-center gap-2 px-3">
<button
onClick={() => setZoom(Math.max(0.25, zoom - 0.25))}
className="w-7 h-7 flex items-center justify-center text-gray-500 hover:text-gray-700 border border-gray-300 rounded text-sm font-bold"
>
−
</button>
<span className="text-xs text-gray-500 w-10 text-center">{Math.round(zoom * 100)}%</span>
<button
onClick={() => setZoom(Math.min(1.5, zoom + 0.25))}
className="w-7 h-7 flex items-center justify-center text-gray-500 hover:text-gray-700 border border-gray-300 rounded text-sm font-bold"
>
+
</button>
{zoom !== 1 && (
<button
onClick={() => setZoom(1)}
className="text-xs text-gray-500 hover:text-gray-700 px-2 py-1 border border-gray-300 rounded"
>
Reset
</button>
)}
</div>
);
}
import { downloadReport } from "../api/reports";
import HeatmapOverlay from "../components/analysis/HeatmapOverlay";
import HeatmapControls from "../components/analysis/HeatmapControls";
import GazeSequence from "../components/analysis/GazeSequence";
import HotspotList from "../components/analysis/HotspotList";
import InsightsPanel from "../components/analysis/InsightsPanel";
import AOICanvas from "../components/aoi/AOICanvas";
import AOIResults from "../components/aoi/AOIResults";
import Card from "../components/common/Card";
import Button from "../components/common/Button";
import LoadingSpinner from "../components/common/LoadingSpinner";
const tabs: { key: AnalysisTab; label: string }[] = [
{ key: "heatmap", label: "Heatmap" },
{ key: "gaze", label: "Gaze Sequence" },
{ key: "hotspots", label: "Hotspots" },
{ key: "aoi", label: "AOI Analysis" },
];
export default function AnalysisView() {
const { analysisId } = useParams<{ analysisId: string }>();
const { data: analysis, isLoading, error } = useAnalysis(analysisId);
const activeTab = useAnalysisStore((s) => s.activeTab);
const setActiveTab = useAnalysisStore((s) => s.setActiveTab);
const setCurrentAnalysis = useAnalysisStore((s) => s.setCurrentAnalysis);
const aoiResults = useAnalysisStore((s) => s.aoiResults);
const reset = useAnalysisStore((s) => s.reset);
const [aiAvailable, setAiAvailable] = useState(false);
const [aiInsights, setAiInsights] = useState<Insight[]>([]);
const [aiCostUsd, setAiCostUsd] = useState<number | null>(null);
const [aiLoading, setAiLoading] = useState(false);
const [aiError, setAiError] = useState<string | null>(null);
useEffect(() => {
checkAIInsightsAvailable().then(setAiAvailable);
}, []);
// Load saved AI insights from DB when analysis loads
useEffect(() => {
if (analysis?.ai_insights && analysis.ai_insights.length > 0) {
setAiInsights(analysis.ai_insights);
setAiCostUsd(analysis.ai_cost_usd ?? null);
}
}, [analysis]);
const handleGenerateAI = async () => {
if (!analysisId) return;
setAiLoading(true);
setAiError(null);
try {
const result = await generateAIInsights(analysisId);
setAiInsights(result.insights);
setAiCostUsd(result.cost_usd);
} catch (err: any) {
setAiError(err?.response?.data?.detail || "AI analysis failed");
} finally {
setAiLoading(false);
}
};
useEffect(() => {
if (analysis) {
setCurrentAnalysis(analysis);
}
return () => {
reset();
};
}, [analysis, setCurrentAnalysis, reset]);
const handleDownloadPdf = async () => {
if (!analysisId) return;
try {
await downloadReport(analysisId);
} catch (err) {
console.error("Failed to download report:", err);
}
};
if (isLoading) {
return <LoadingSpinner size="lg" message="Loading analysis..." />;
}
if (error || !analysis) {
return (
<div className="text-center py-12">
<p className="text-red-500 mb-4">
Failed to load analysis. It may not exist or is still processing.
</p>
<Button variant="secondary" onClick={() => window.history.back()}>
Go Back
</Button>
</div>
);
}
const originalUrl = getAnalysisImageUrl(analysis.id, "original");
const saliencyUrl = getAnalysisImageUrl(analysis.id, "saliency-raw");
const gazeUrl = getAnalysisImageUrl(analysis.id, "gaze-sequence");
return (
<div className="max-w-6xl mx-auto space-y-6">
{/* Header */}
<div className="flex flex-col sm:flex-row sm:items-center justify-between gap-4">
<div>
<h1 className="text-2xl font-bold" style={{ color: "#1a1a2e" }}>
{analysis.name}
</h1>
<div className="flex items-center gap-4 mt-1 text-sm text-gray-500">
<span>Model: {analysis.model_used || analysis.model}</span>
<span>
{new Date(analysis.created_at).toLocaleDateString()}
</span>
{analysis.overall_score !== undefined && (
<span
className="font-semibold px-2 py-0.5 rounded"
style={{
backgroundColor: "#ffc40720",
color: "#ffc407",
}}
>
Score: {analysis.overall_score.toFixed(1)}
</span>
)}
</div>
</div>
<div className="flex items-center gap-3">
{aiAvailable && analysis.status === "completed" && aiInsights.length === 0 && (
<Button
variant="secondary"
onClick={handleGenerateAI}
loading={aiLoading}
>
Generate AI Analysis
</Button>
)}
<Button onClick={handleDownloadPdf}>Download PDF</Button>
</div>
</div>
{/* Tabs + Zoom */}
<div className="border-b border-gray-200">
<div className="flex items-center justify-between">
<div className="flex">
{tabs.map((tab) => (
<button
key={tab.key}
onClick={() => setActiveTab(tab.key)}
className={`px-5 py-3 text-sm font-medium border-b-2 transition-colors ${
activeTab === tab.key
? "border-[#ffc407] text-[#ffc407]"
: "border-transparent text-gray-500 hover:text-gray-700 hover:border-gray-300"
}`}
>
{tab.label}
</button>
))}
</div>
<ZoomControls />
</div>
</div>
{/* Insights */}
{((analysis.insights && analysis.insights.length > 0) || aiInsights.length > 0) && (
<InsightsPanel insights={analysis.insights || []} aiInsights={aiInsights} aiCostUsd={aiCostUsd} />
)}
{aiError && (
<div className="text-sm text-red-500 bg-red-50 border border-red-200 rounded-lg px-4 py-2">
{aiError}
</div>
)}
{/* Tab content */}
{activeTab === "heatmap" && (
<div className="grid grid-cols-1 lg:grid-cols-4 gap-6">
<div className="lg:col-span-3">
<Card padding={false} className="overflow-auto p-2">
<HeatmapOverlay
originalUrl={originalUrl}
saliencyUrl={saliencyUrl}
/>
</Card>
</div>
<div className="lg:col-span-1">
<HeatmapControls />
</div>
</div>
)}
{activeTab === "gaze" && (
<GazeSequence
imageUrl={gazeUrl}
gazePoints={
(analysis.gaze_sequence || analysis.gaze_points || []).map((p: any) => ({
rank: p.rank,
x: p.x,
y: p.y,
intensity: p.probability ?? p.intensity ?? 0,
}))
}
/>
)}
{activeTab === "hotspots" && (
<div className="grid grid-cols-1 lg:grid-cols-2 gap-6">
<Card padding={false} className="overflow-auto">
<ZoomableImage
src={getAnalysisImageUrl(analysis.id, "heatmap")}
alt="Heatmap with hotspots"
/>
</Card>
<HotspotList hotspots={analysis.hotspots || []} />
</div>
)}
{activeTab === "aoi" && (
<div className="space-y-6">
<AOICanvas analysisId={analysis.id} imageUrl={originalUrl} />
<AOIResults results={aoiResults} />
</div>
)}
</div>
);
}

175
frontend/src/pages/ComparisonView.tsx Normal file
View file

@ -0,0 +1,175 @@
import { useEffect, useState } from "react";
import { useParams, useNavigate } from "react-router-dom";
import { useProject } from "../hooks/useProjects";
import { getAnalysisImageUrl } from "../api/analysis";
import Card from "../components/common/Card";
import Button from "../components/common/Button";
import LoadingSpinner from "../components/common/LoadingSpinner";
export default function ComparisonView() {
const { comparisonId } = useParams<{ comparisonId: string }>();
const navigate = useNavigate();
const { data: project, isLoading } = useProject(comparisonId);
const [selectedIds, setSelectedIds] = useState<string[]>([]);
const analyses = (project?.analyses || []).filter(
(a) => a.status === "completed",
);
useEffect(() => {
if (analyses.length >= 2 && selectedIds.length === 0) {
setSelectedIds(analyses.slice(0, 2).map((a) => a.id));
}
}, [analyses, selectedIds.length]);
const toggleSelection = (id: string) => {
setSelectedIds((prev) =>
prev.includes(id) ? prev.filter((i) => i !== id) : [...prev, id],
);
};
if (isLoading) {
return <LoadingSpinner size="lg" message="Loading comparison..." />;
}
if (!project) {
return (
<div className="text-center py-12">
<p className="text-red-500 mb-4">Project not found.</p>
<Button variant="secondary" onClick={() => navigate("/")}>
Back to Dashboard
</Button>
</div>
);
}
const selected = analyses.filter((a) => selectedIds.includes(a.id));
return (
<div className="max-w-7xl mx-auto space-y-6">
<div>
<h1 className="text-2xl font-bold" style={{ color: "#1a1a2e" }}>
Compare Analyses
</h1>
<p className="text-gray-500 text-sm mt-1">
{project.name} -- Side-by-side heatmap comparison
</p>
</div>
{/* Selection */}
<Card>
<h3 className="text-sm font-semibold text-gray-700 mb-3">
Select analyses to compare:
</h3>
<div className="flex flex-wrap gap-2">
{analyses.map((a) => (
<button
key={a.id}
onClick={() => toggleSelection(a.id)}
className={`px-3 py-1.5 rounded-lg text-sm border transition-colors ${
selectedIds.includes(a.id)
? "border-[#ffc407] bg-[#ffc407]/10 text-[#ffc407] font-medium"
: "border-gray-300 text-gray-600 hover:border-gray-400"
}`}
>
{a.name}
</button>
))}
</div>
</Card>
{/* Side-by-side view */}
{selected.length >= 2 ? (
<div
className="grid gap-6"
style={{
gridTemplateColumns: `repeat(${Math.min(selected.length, 3)}, 1fr)`,
}}
>
{selected.map((analysis) => (
<Card key={analysis.id} padding={false} className="overflow-hidden">
<div className="p-3 border-b border-gray-100">
<h4
className="font-medium text-sm truncate"
style={{ color: "#1a1a2e" }}
>
{analysis.name}
</h4>
<p className="text-xs text-gray-400">{analysis.model}</p>
</div>
<div className="aspect-video bg-gray-100">
<img
src={getAnalysisImageUrl(analysis.id, "heatmap")}
alt={`Heatmap: ${analysis.name}`}
className="w-full h-full object-contain"
/>
</div>
</Card>
))}
</div>
) : (
<Card className="text-center py-8">
<p className="text-gray-400">
Select at least 2 completed analyses to compare.
</p>
</Card>
)}
{/* Comparison table */}
{selected.length >= 2 && (
<Card>
<h3 className="text-sm font-semibold text-gray-700 uppercase tracking-wide mb-4">
Comparison Metrics
</h3>
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead>
<tr className="border-b border-gray-200">
<th className="text-left py-2 px-3 text-gray-500 font-medium">
Metric
</th>
{selected.map((a) => (
<th
key={a.id}
className="text-right py-2 px-3 text-gray-500 font-medium"
>
{a.name}
</th>
))}
</tr>
</thead>
<tbody>
<tr className="border-b border-gray-100">
<td className="py-2 px-3 text-gray-700">Model</td>
{selected.map((a) => (
<td key={a.id} className="py-2 px-3 text-right text-gray-600">
{a.model}
</td>
))}
</tr>
<tr className="border-b border-gray-100">
<td className="py-2 px-3 text-gray-700">Status</td>
{selected.map((a) => (
<td key={a.id} className="py-2 px-3 text-right">
<span className="text-green-600 font-medium">
{a.status}
</span>
</td>
))}
</tr>
<tr className="border-b border-gray-100">
<td className="py-2 px-3 text-gray-700">Date</td>
{selected.map((a) => (
<td key={a.id} className="py-2 px-3 text-right text-gray-600">
{new Date(a.created_at).toLocaleDateString()}
</td>
))}
</tr>
</tbody>
</table>
</div>
</Card>
)}
</div>
);
}

Some files were not shown because too many files have changed in this diff Show more