agent_tracker/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

AgentHub is a FastAPI-based AI Agent Management System with MongoDB backend. It provides three-tier role-based authentication (admin/readonly_admin/user), agent CRUD operations, client verification workflow, email notifications, user management, and a web interface built with Jinja2 templates and Bootstrap 5.

## Common Development Tasks

### Running the Application
```bash
uvicorn main:app --reload --port 8000
```
Access at: http://localhost:8000

### Installing Dependencies
```bash
pip install -r requirements.txt
```

### Environment Setup
Create `.env` file with:
- `MONGODB_URI`: MongoDB connection string (default: mongodb://localhost:27017)
- `MONGODB_DBNAME`: Database name (default: agenthub_db)
- `SECRET_KEY`: JWT secret key
- `ALGORITHM`: JWT algorithm (default: HS256)
- `ACCESS_TOKEN_EXPIRE_MINUTES`: Token expiration (default: 60)

#### Optional: Prompt Audit (Gemini)
- `GOOGLE_API_KEY`: Google API key for Gemini (if not set, audit is silently disabled)
- `AUDIT_GEMINI_MODEL`: Gemini model name (default: `gemini-2.5-pro`)
- `AUDIT_CONCURRENCY`: Batch size for sequential processing (default: 2)

#### Optional: Email Notifications (Mailgun)
- `MAILGUN_API_KEY`: Mailgun API key (if not set, notifications are silently disabled)
- `MAILGUN_DOMAIN`: Mailgun sending domain (e.g. your-domain.mailgun.org)
- `MAILGUN_FROM_EMAIL`: Sender address (default: `AgentHub <noreply@{MAILGUN_DOMAIN}>`)
- `NOTIFICATION_REPLY_TO`: Reply-To address attached to every Mailgun send (default: `Nick.Viljoen@oliver.agency`). Lets recipients reply to a real mailbox instead of the `noreply@` sender.
- `TOKEN_USAGE_THRESHOLD`: Weekly token count that triggers an alert (default: 100000)
- `NOTIFICATION_COOLDOWN_HOURS`: Hours between repeat alerts for the same agent (default: 24)
- `CLIENT_AGENT_NOTIFY_EMAILS`: Comma-separated list of emails for client agent notifications
- `WEEKLY_DIGEST_HOUR`: Hour (24h format) to send weekly digest on Mondays (default: 7)

#### Optional: Completion-Reminder Emails (LibreChat completion flow)
- `COMPLETION_REMINDER_COOLDOWN_DAYS`: Min days between reminders for the same user (default: 7)
- `COMPLETION_REMINDER_MAX_NUDGES`: After this many nudges, stop emailing the user (default: 4)
- `COMPLETION_REMINDER_HOUR`: Hour (24h) for the daily reminder cron (default: 8)
- `AGENTHUB_PUBLIC_URL`: Public absolute URL used in email links (e.g. `https://agenthub.example.com`). If unset, links are relative and won't work in email clients.

### Default Login Credentials
- Admin: `admin@agenthub.com` / `admin123`
- Test User: `test@example.com` / `testpass123`

## Code Architecture

### Core Application Structure

**main.py**: FastAPI application with:
- JWT cookie-based authentication system
- HTML routes for web interface
- REST API endpoints for agent/user management
- Three-tier role-based access control (admin / readonly_admin / user)
- Client verification workflow with email notifications
- Daily agent digest scheduler (APScheduler)
- Template rendering with Jinja2

**Key Authentication Functions**:
- `get_current_user_optional()`: Cookie-based auth for templates
- `get_current_user_from_cookie()`: Required auth for API endpoints
- `require_admin()`: Admin-only access control (write operations)
- `require_admin_or_readonly()`: Admin + readonly_admin access (read-only dashboard views)

### Data Layer

**models.py**: Pydantic models for:
- `AiAgent`: Core agent model with comprehensive fields (includes `discipline`, `rating`, `client`, `client_name`, `studio_name`)
- `UsageTimelineEntry`: Daily usage data including `message_count` and `token_count`
- `UserCreate/UserResponse`: User management models (includes `role` field: `user`/`admin`/`readonly_admin`)
- `UserUpdate`: Includes `role` field for three-tier role management
- `AiAgentCreate/AiAgentResponse`: API request/response models (includes `total_tokens`, `prompt_tokens`, `completion_tokens`, `discipline`, `rating`, `rating_count`, `client`, `client_name`, `studio_name`, `verification_status`, `verified_by`, `verified_date`)
- `AgentCollectorCreate`: Collector API input model (includes `total_tokens`, `prompt_tokens`, `completion_tokens`, `discipline`, `client`, `client_name`, `studio_name`)
- `AuditReviewRequest`: Audit review input (`audit_status`: flagged/reviewed/cleared, `reviewer_notes`)
- `AgentUsageStatsResponse`: Usage statistics response (includes `total_tokens`, `prompt_tokens`, `completion_tokens`)

**crud.py**: Database operations using Motor (async MongoDB driver):
- User CRUD: authentication, creation, management
- Agent CRUD: create, read, update, delete with user ownership
- Advanced features: search, filtering, statistics, pagination
- All operations use ObjectId for MongoDB document IDs

**database.py**: MongoDB connection setup with Motor async client
- Collections: `users`, `agents`, `agent_usage`, `token_notifications`, `agent_ratings`, `audit_history`, `completion_reminders`
- `ensure_indexes()`: Creates compound unique index on `agent_ratings(agent_id, user_id)`, indexes on `verification_status`, `audit_status`, and `audit_history(agent_id, audit_date)`

**audit_analyzer.py**: Gemini-powered agent classification and audit system:
- `is_gemini_configured()`: Returns False if `GOOGLE_API_KEY` not set (gracefully disabled)
- `analyze_single_agent()`: Sends agent instructions to Gemini 2.5 Pro, returns structured JSON with category, discipline, department, client detection, risk level, flags, and recommendations. Includes retry with exponential backoff for rate limits.
- `store_audit_result()`: Writes audit results to agent document (`audit_status`, `audit_category`, `audit_risk_level`, etc.) and inserts into `audit_history` collection
- `apply_classification_fields()`: Auto-assigns `discipline` (from defined list), `agent_department` (free text inferred from instructions), and client detection (`client = "yes"`, `verification_status = "needs_verification"`) — only overwrites fields that are currently empty/null
- `classify_single_agent()`: Convenience function for post-sync automatic classification. Loads agent, analyses, stores result, applies fields.
- `run_audit_batch()`: Batch processes agents sequentially with rate-limit-safe pauses. Supports `unclassified_only` and `single_agent_id` params. Returns summary with audited/failed/skipped counts.
- `get_all_audit_results()`: Returns all agents with audit fields for the Prompt Audit tab
- `update_audit_review()`: Admin marks audit as reviewed/cleared with notes
- Uses `google-genai` SDK (new API), model configurable via `AUDIT_GEMINI_MODEL` env var (default: `gemini-2.5-pro`)

**notifications.py**: Mailgun email notification system:
- `is_mailgun_configured()`: Returns False if env vars not set (gracefully disabled)
- `send_mailgun_email()`: POST to Mailgun HTTP API with 10s timeout
- `build_threshold_email()`: HTML email template for threshold alerts
- `check_and_notify_threshold()`: Checks 7-day token usage against threshold, enforces cooldown via `token_notifications` collection, sends to admin users
- `send_client_agent_notification()`: Sends email when client-facing agent is created (to `CLIENT_AGENT_NOTIFY_EMAILS`)
- `build_client_agent_email()`: HTML email template for client agent notifications
- `send_weekly_agent_digest()`: Queries agents created in last 7 days, sends summary to all admin users
- `build_weekly_digest_email()`: HTML email template for weekly digest

**auth.py**: JWT authentication with:
- bcrypt password hashing
- JWT token creation/validation using python-jose
- Configurable token expiration

### Frontend Templates

Located in `templates/` directory:
- **base.html**: Bootstrap 5 base template with navigation
- **nav.html**: Dynamic navigation based on user role
- **index.html**: Landing page
- **login.html/register.html**: Authentication forms
- **agent_register.html**: Agent creation form
- **agent_management.html**: Agent dashboard with real data
- **search.html**: Global search functionality
- **user_management.html**: User management interface
- **admin/dashboard.html**: Admin statistics, management, and verification workflow

### Static Assets

**static/style.css**: Custom CSS with:
- CSS variables for consistent theming
- Gradient backgrounds and modern styling
- Responsive design for mobile devices
- Bootstrap 5 customizations

## Key Features

### Authentication Flow
- Cookie-based JWT authentication
- Three-tier role-based access: `user`, `admin`, `readonly_admin`
  - `user`: Standard access, can manage own agents
  - `admin`: Full access to admin dashboard, all write operations
  - `readonly_admin`: Can view admin dashboard but all write actions (edit, delete, approve, create) are hidden
- `role` field on user documents; `is_admin` kept in sync for backward compatibility
- `require_admin_or_readonly()` dependency for read-only admin endpoints
- Automatic redirects based on user role (admin/readonly_admin → /admin, user → /agent-management)
- Secure logout with token cleanup

### Agent Management
- Full CRUD operations with user ownership
- Status tracking (Active, Inactive, Development, Deprecated)
- Rich metadata: tags, userbase, department, contact person
- Search functionality across multiple fields
- Filtering by status, audit status (Audited / Not Audited), and discipline
- Admin can view/manage all agents

### Client & Verification System
- `client` field on agents: `"yes"` or `"no"` (mandatory on registration form)
- `client_name`: free text, required when client is "yes"
- `studio_name`: optional free text field
- Registration form order: Name, Description, Purpose, Client (Yes/No), Client Name (conditional), Studio Name, Tool, then Version/Status/etc.
- When `client == "yes"`: agent auto-tagged with `verification_status = "needs_verification"`
- Verification tab on admin dashboard shows pending agents with Approve button
- `PUT /api/admin/agents/{id}/verify` — admin-only, sets status to "verified" with verifier info
- `GET /api/admin/agents/pending-verification` — returns agents needing verification
- Verification badges displayed on agent cards (orange "Needs Verification", green "Verified")

### Client Agent Email Notification
- When `client == "yes"` on agent creation, sends email via Mailgun to `CLIENT_AGENT_NOTIFY_EMAILS`
- Subject: "Client Agent Created"
- Body includes: Agent Name, Description, Purpose, Client Name, Studio Name, Tool, Created By
- Non-blocking: failure does not break agent creation

### Weekly Agent Digest Email
- Scheduled via APScheduler to run every Monday morning (default 7:00 AM, `WEEKLY_DIGEST_HOUR` env var)
- Queries agents created in last 7 days
- Sends to all active admin users
- Body includes: Agent Name, Purpose, Description, Created By (email)
- Subject: "Agents Created in Last Week"
- Skips sending if no agents created
- Can be manually triggered via `POST /api/admin/digest/send` (admin-only)

### Discipline & Star Rating
- `discipline` field classifies agents into business categories: Strategy, Creative, Oversight including delivery, Optimization, Back Office including operations, Pencil Agents
- Required on registration form, optional on edit (to support legacy agents)
- Pencil Agents discipline is auto-assigned to agents with "pencil" in the name when no discipline is set (collector API auto-tag + startup migration)
- `rating` field stores the **average** star rating (1-5) computed from all per-user ratings
- `rating_count` field stores the number of individual ratings
- **Per-user rating system**: Any authenticated user can rate any agent via `PUT /api/agents/{id}/rating`
  - Individual ratings stored in `agent_ratings` collection with compound unique index on `(agent_id, user_id)`
  - After each rating, the agent's average rating and count are recalculated and stored on the agent document
  - `GET /api/agents/{id}/my-rating` returns the current user's rating plus the average and count
- Interactive star rating widget in detail modal shows the **user's own rating** as filled stars
- Average rating and count displayed below the stars and on agent card badges
- Rating removed from edit modals (rating is per-user, not admin-set)
- Rating framework info modal accessible via info icon next to "Rating:" label
- Dashboard supports filtering by discipline and sorting by rating
- Discipline badge (purple) and star rating badge displayed on agent cards
- Both fields included in CSV export/import
- Discipline passed through collector API; rating is human-only (not in collector)

### Token Usage Tracking
- `total_tokens`, `prompt_tokens`, `completion_tokens` fields on agents track cumulative LLM token consumption
- `prompt_tokens` (input) and `completion_tokens` (output) provide cost breakdown detail
- `token_count` per day in usage timeline entries alongside `message_count`
- Token badge displayed on agent cards (gold/coins icon) with prompt/completion breakdown in tooltip
- Usage modal shows Total Tokens stat with In/Out breakdown alongside messages/conversations/users
- Dual-axis chart (messages left axis, tokens right axis) when token data exists
- Sort agents by Total Tokens
- CSV export includes `total_tokens`, `prompt_tokens`, `completion_tokens` columns

### High Usage Email Notifications
- Entirely optional — silently disabled when Mailgun env vars are not set
- Triggered from the Agent Collector endpoint (POST `/agents`); checks 7-day rolling token usage from `usage_timeline`
- Alerts when weekly token usage exceeds threshold (default 100,000, configurable via `TOKEN_USAGE_THRESHOLD`)
- Non-blocking — notification failure never breaks the collector API
- Cooldown tracking in MongoDB `token_notifications` collection (default 24h, configurable)
- Sends to all active admin users' email addresses

### Prompt Audit & Auto-Classification (Gemini)
- Automated analysis of agent `instructions` (system prompts) using Google Gemini 2.5 Pro
- **Two trigger modes:**
  1. **Automatic post-sync**: After the collector API (`POST /agents`) creates/updates an agent with instructions, a background task (`asyncio.create_task`) auto-classifies it. Non-blocking — sync response is not delayed.
  2. **Manual batch**: Admin clicks "Run Full Audit" or "Run Unclassified Only" on the Prompt Audit tab. Processes agents sequentially with 4-second pauses between batches to avoid Gemini rate limits.
- **Classification outputs per agent:**
  - `audit_category`: Cat 1 (Internal Sandbox), Cat 1B (High Cost Internal), Cat 2 (Client-Exposed), Cat 3 (Client-Sold)
  - `audit_risk_level`: low / medium / high / critical
  - `audit_discipline`: Picks from existing discipline list (Strategy, Creative, Oversight including delivery, Optimization, Back Office including operations, Pencil Agents)
  - `audit_department`: Free text inferred from instructions (e.g., "Project Management", "Media")
  - `audit_is_client_work`: Boolean — detects client names, brands, client deliverables in instructions
  - `audit_flags`: Array of risk flags (client_facing, handles_pii, uses_external_tools, etc.)
  - `audit_summary`, `audit_recommendations`, `audit_category_reasoning`, `audit_discipline_reasoning`, `audit_client_work_reasoning`, `audit_client_name_detected`
- **Auto-assignment**: Gemini results auto-populate `discipline` and `agent_department` fields on the agent document (only if currently empty, to respect manual edits)
- **Client work auto-detection**: When `audit_is_client_work = true`, auto-sets `client = "yes"` and `verification_status = "needs_verification"` — agent appears on Verification tab. Does NOT overwrite manually-set values.
- **Audit statuses**: All audited agents start as `flagged`. Admin can mark as `reviewed` or `cleared` via the detail modal with optional notes.
- **Prompt Audit tab** on admin dashboard: Summary cards (Audited, Flagged, Reviewed, Cleared, Client Detected, No Instructions), filterable results table, detail modal with full analysis and review controls
- **Rate limit handling**: Retry with exponential backoff (10s, 20s, 40s) for 429/quota errors
- **Logging**: Uses Python `logging` module (`audit_analyzer` logger) for systemd journal visibility
- Agents without `instructions` are skipped (counted as "No Instructions")
- `audit_history` collection stores historical record of each audit run and review action
- Gracefully disabled when `GOOGLE_API_KEY` not set — UI shows config warning, buttons hidden

### User Management
- User registration with validation
- Admin user creation capabilities
- Three-tier role system: `user`, `admin`, `readonly_admin`
  - Role dropdown in admin user edit modal (replaces is_admin checkbox)
  - `role` and `is_admin` fields kept in sync on update
- Profile management
- User statistics and administration

### Database Integration
- MongoDB with proper ObjectId handling
- Async operations using Motor driver
- Indexed queries for performance
- Data aggregation for statistics
- Collections: `users`, `agents`, `agent_usage`, `token_notifications`, `agent_ratings`, `audit_history`, `completion_reminders`

## Development Guidelines

### Database Operations
- Always use ObjectId for MongoDB document IDs
- Use Motor async driver methods (await collection.find_one())
- Handle ObjectId conversion in CRUD operations
- Implement proper error handling with try/except blocks

### Authentication
- Use cookie-based auth for web interface
- API endpoints require `get_current_user_from_cookie()` dependency
- Write endpoints use `require_admin()` dependency
- Read-only admin endpoints use `require_admin_or_readonly()` dependency
- Always validate user permissions for data access
- `readonly_admin` users can view admin dashboard but UI hides all write-action buttons via `admin-write-action` CSS class

### Template Context
- Pass `current_user` to all templates for navigation
- Handle dict objects (not User model instances) in templates
- Use proper null checks for optional user data

### API Response Models
- Convert ObjectId to string in API responses
- Handle optional datetime fields with isoformat()
- Maintain consistency between Create and Response models

### Error Handling
- Provide meaningful error messages in templates
- Use proper HTTP status codes in API responses
- Graceful degradation for missing data

## Project Dependencies

Key dependencies from requirements.txt:
- **fastapi**: Web framework
- **uvicorn**: ASGI server
- **motor**: Async MongoDB driver
- **pymongo**: MongoDB operations
- **python-jose**: JWT token handling
- **passlib**: Password hashing
- **bcrypt**: Password encryption
- **pydantic**: Data validation
- **jinja2**: Template engine
- **python-multipart**: Form handling
- **requests**: HTTP client (used for Mailgun API calls)
- **apscheduler**: Task scheduling (weekly digest email)
- **google-genai**: Google Gemini API client (used for prompt audit auto-classification)

## API Endpoints (New)

### Verification
- `GET /api/admin/agents/pending-verification` — List agents with verification status (admin + readonly_admin)
- `PUT /api/admin/agents/{agent_id}/verify` — Approve/verify an agent (admin only)

### Weekly Digest
- `POST /api/admin/digest/send` — Manually trigger the weekly agent digest email (admin only)

### Prompt Audit
- `POST /api/admin/audit/run` — Run Gemini audit batch. Optional JSON body: `{agent_id, unclassified_only}`. Returns `{status, total, audited_count, failed_count, skipped_count, results_summary}` (admin only)
- `GET /api/admin/audit/results` — Get all agents with audit data + `config_status.gemini_configured` (admin + readonly_admin)
- `PUT /api/admin/audit/{agent_id}/review` — Mark audit as reviewed/cleared with `{audit_status, reviewer_notes}` (admin only)

### Registration Completion Flow (LibreChat-synced agents)
- `GET /api/agents/incomplete` — Current user's incomplete agents (admins see all). Used by the agent management page banner. Owner resolution: `created_by == user_id` OR (`created_by == "agent_collector_api"` AND `agent_contact_person == user_email`).
- `GET /agent-complete/{agent_id}` — HTML form (parameterised `agent_register.html` in completion mode). Pre-fills existing fields, required new governance fields blank. Auth: agent owner (by ID or email) or admin.
- `POST /agent-complete/{agent_id}` — Submit completion. Sets `registration_complete=True`, reassigns `created_by` from `"agent_collector_api"` to the submitting user's ID.
- `POST /api/admin/completion-reminders/send` — Manually trigger the daily reminder digest. Optional `?force=true` to bypass cooldown + max-nudges cap (admin only).
- `GET /api/admin/agents/unresolved-owner` — Collector-created agents whose `agent_contact_person` doesn't match an active user (admin only)
- `PUT /api/admin/agents/{agent_id}/reassign-owner` — Admin reassigns ownership; body `{new_contact_email, new_owner_user_id?}`. Resolves user by email if `new_owner_user_id` omitted (admin only)

## Registration Form Redesign (2026-05)

The agent registration form was rebuilt around 7 governance sections per `Indext2.html` mockup. See `PLAN-registration-form-redesign.md` for full design rationale.

### New schema fields on agents
Top-level (all `Optional`):
- `business_entity` (enum: OLIVER, DARE, Brandtech Group, Pencil, Jellyfish, Adjust, Other) — required on form
- `client_scope` (enum: `internal`, `all`, `specific`) — required on form. Maps to legacy `client` field: `specific`→`yes`, else→`no`. Both fields stay in sync to preserve existing verification + Mailgun client-agent-email flows.
- `agent_classification` (enum: Utility, Functional, Supervisory, Guardian) — required on form
- `autonomy_level` (enum: Human-Led, Hybrid, Autopilot) — required on form
- `ip_ownership` (enum: Brandtech IP, Client IP, Shared/TBD) — required on form
- `foundation_model` (string) — optional, dropdown with optgroups by provider
- `validated_by`, `validation_date`, `evals_method` — optional Performance & Testing fields
- `registration_complete` (bool) — `True` for form-submitted agents, `False` for collector-created. Drives the completion-reminder flow.

Nested objects:
- `safety: { off_switch_confirmed, access_rights_confirmed }` — booleans, recorded but don't block submission
- `pii: { handles_pii, legal_ref, data_types, consent_recorded }` — Yes/No radio with conditional fields
- `declarations: { governance, accuracy, upkeep }` — three required checkboxes

Module-level enum constants live in `models.py` (`BUSINESS_ENTITIES`, `CLIENT_SCOPES`, `AGENT_CLASSIFICATIONS`, `AUTONOMY_LEVELS`, `IP_OWNERSHIPS`, `DISCIPLINES`).

### Form field mapping
- "Studio / Department" form input → stored as `studio_name` (not `agent_department` — `agent_department` continues to be auto-populated by Gemini audit)
- Author / Contact Person — autofilled from logged-in user's email; not a user-editable input
- Capabilities — 8 fixed checkboxes ∪ comma-split values from "Other" free-text input, stored as `agent_capabilities: list[str]`

### Discipline rename
Discipline value `Optimization` (US) was renamed to `Optimisation` (UK) in 2026-05. Migration runs on startup (`crud.migrate_optimisation_spelling`). All template dropdowns + Gemini system prompt updated. Defensive `Optimization → Optimisation` normalisation in `audit_analyzer.store_audit_result` and `apply_classification_fields`.

### Required-field enforcement
The `/agent-register` POST handler enforces required fields server-side via `Form(...)` defaults plus explicit conditional checks (e.g. `client_name` required if `client_scope == "specific"`, all 3 declarations required). Existing agents are grandfathered via the startup migration that backfilled `registration_complete` based on `created_by`.

### Completion flow for LibreChat-synced agents
LibreChat-synced agents enter via the collector with `created_by="agent_collector_api"` and only a subset of fields populated. They land with `registration_complete=False` (set in `crud.create_agent_from_collector` via `setdefault`). Owners are nudged via daily APScheduler email job (`notifications.send_completion_reminders`):

1. Job groups incomplete agents by lowercased `agent_contact_person` and resolves to active users (case-insensitive email match)
2. Per-user cooldown (default 7 days) + nudge cap (default 4) tracked in `completion_reminders` collection
3. Single digest email per user with one "Complete →" CTA per agent
4. Submitting the completion form flips `registration_complete=True` AND reassigns `created_by` from the marker to the submitting user's ID — they then own it normally for future edits

`agent_register.html` is parameterised: `mode="register"` (default) or `mode="complete"`. The completion-mode handler `_build_completion_context()` in `main.py` pre-computes capability sets, derived `client_scope` from legacy `client` field, and PII/safety pre-fills. Special case: `agent_purpose` renders blank if it equals `agent_description` (collector defaults purpose=description for ~94% of agents per prod data analysis).

### Gemini audit auto-classification (extended 2026-05)
`audit_analyzer.SYSTEM_INSTRUCTION` schema extended to also return:
- `agent_classification` (Utility/Functional/Supervisory/Guardian) — auto-applied to `agent_classification` field if empty
- `autonomy_level_hint` (Human-Led/Hybrid/Autopilot/null) — auto-applied to `autonomy_level` if empty
- `agent_classification_reasoning`, `autonomy_reasoning` — stored on agent doc as audit fields (not used as primary values)

Running `POST /api/admin/audit/run` after deploy auto-fills these on the ~225 agents with instructions, leaving only the truly non-inferable fields (business_entity, ip_ownership, declarations) for the human owner via the completion form.

### Filter dimensions
Both `agent_management.html` and `admin/dashboard.html` agent views support filtering by:
- Business Entity, Agent Type, Autonomy Level (new dropdowns)
- "Compliance risks" quick toggle — shows agents where `pii.handles_pii=true` OR `ip_ownership="Shared/TBD"` OR `autonomy_level="Autopilot"`

### CSV roundtrip for backfilling
`/api/admin/agents/export/csv` includes 21 new columns covering all governance fields. Import accepts them; nested objects (`safety`, `pii`, `declarations`) only build up if at least one cell is populated, so partially-filled CSVs don't clobber existing data with `False`. Defensive `_csv_bool()` helper returns `None` for empty cells.

## UI & Brand (2026-05)

The frontend was rebranded to the OLIVER master template (`/Users/nickviljoen/Documents/Optical_Projects/Pres Template/OLIVER Master PPT Template.potx`). The old muddy-orange `#f3ae3e` palette and Inter font are gone. See `PLAN-ui-refresh.md` for design rationale.

### Design tokens (`static/style.css` `:root`)
Always use these tokens — do **not** introduce new hex codes:
- `--brand-yellow: #FFCB05` — primary accent (page-title highlight, stat-tile left strip, card-header icons, active states, primary buttons)
- `--brand-yellow-soft: #FFF5C4` — subtle hover backgrounds, dropdown-item hover
- `--brand-orange: #FF5C00` — destructive accent (btn-danger, Microsoft SSO button, logout link)
- `--brand-dark: #1A1A1A` — body text, table headers, modal headers, nav text
- `--brand-grey: #626262` — secondary text / muted copy
- `--brand-grey-light: #DEE2E5` — borders, dividers
- `--brand-off-white: #F6F7F7` — page background, table row hover
- Font: **Montserrat** 400/500/600/700/800 (loaded from Google Fonts in `base.html`)

### Signature motifs
- **Page title** — `.page-title` class. Yellow highlight rectangle behind black text via a `linear-gradient` (yellow up to 70% of the height, transparent above). Apply to every page `<h2>`; subtitle goes in `<p class="page-subtitle">` below. **Don't** wrap with a leading icon — the highlight is the visual.
- **Stat tile** — `.stat-tile` class. White card with 8px yellow `::before` strip on the left, yellow Font Awesome icon, big near-black `.stat-tile-value` and uppercase `.stat-tile-label`. Replaced the old orange-gradient `.stat-card`.
- **Card section accent** — every `<h5>` / `<h6>` inside `.card-header` automatically gets a leading yellow icon (CSS targets `.card-header h5 > i.fas:first-child`) and a 28×3px yellow underline accent (`::after`). The accent doesn't apply to modal headers or to header titles displaying counts (those go in `.card-body`).
- **Modal header** — solid `--brand-dark` background, white text, white close button.
- **Active nav-link / nav-tab** — 2-3px `--brand-yellow` bottom border, never a coloured background pill.

### Gotchas (don't re-introduce these bugs)
- **Navbar must use `position: fixed`** (set in `style.css`). Do **not** add `position: relative` or `position: sticky` overrides in `templates/nav.html` — a previous inline `<style>` rule there silently overrode the fixed positioning and made the navbar scroll away with the page.
- **Bootstrap tabs must use `<button data-bs-target="#x">`**, not `<a href="#x">`. The anchor form causes the browser to scroll to the tab pane on click (visible as the page "jumping down"). Bootstrap 5 recommends buttons for this reason.
- **Body has `padding-top: 64px`** to clear the fixed navbar — keep it whenever you change navbar height.
- **No `transform: scale(...)` on table-row hover** — jittery and shifts neighbour rows. Use a background-color change.
- **Chart.js colours** in `templates/admin/dashboard.html` reference the brand palette directly (`#FFCB05`, `#1A1A1A`, `#FF5C00`) because the library can't read CSS vars. Keep them aligned when adding charts.

### Where things live
- `static/style.css` — all tokens + component styles (palette, navbar, tabs, buttons, forms, tables, cards, stat-tile, page-title, modal, alerts).
- `templates/base.html` — Google Fonts link (Montserrat).
- `templates/nav.html` — navbar markup + scoped styles for user-avatar / dropdown.
- `templates/admin/dashboard.html` — inline `<style>` block has admin-specific overrides (column widths, audit risk badge colours, tabs); kept in-file because they're not reused elsewhere.
- Page templates apply `.page-title` to their main `<h2>` and `.page-subtitle` to the description line.