agent_tracker/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

AgentHub is a FastAPI-based AI Agent Management System with MongoDB backend. It provides three-tier role-based authentication (admin/readonly_admin/user), agent CRUD operations, client verification workflow, email notifications, user management, and a web interface built with Jinja2 templates and Bootstrap 5.

Common Development Tasks

Running the Application

uvicorn main:app --reload --port 8000

Access at: http://localhost:8000

Installing Dependencies

pip install -r requirements.txt

Environment Setup

Create .env file with:

• MONGODB_URI: MongoDB connection string (default: mongodb://localhost:27017)
• MONGODB_DBNAME: Database name (default: agenthub_db)
• SECRET_KEY: JWT secret key
• ALGORITHM: JWT algorithm (default: HS256)
• ACCESS_TOKEN_EXPIRE_MINUTES: Token expiration (default: 60)

Optional: Email Notifications (Mailgun)

• MAILGUN_API_KEY: Mailgun API key (if not set, notifications are silently disabled)
• MAILGUN_DOMAIN: Mailgun sending domain (e.g. your-domain.mailgun.org)
• MAILGUN_FROM_EMAIL: Sender address (default: AgentHub <noreply@{MAILGUN_DOMAIN}>)
• TOKEN_USAGE_THRESHOLD: Token count that triggers an alert (default: 100000)
• NOTIFICATION_COOLDOWN_HOURS: Hours between repeat alerts for the same agent (default: 24)
• CLIENT_AGENT_NOTIFY_EMAILS: Comma-separated list of emails for client agent notifications
• DAILY_DIGEST_HOUR: Hour (24h format) to send daily digest (default: 7)

Default Login Credentials

• Admin: admin@agenthub.com / admin123
• Test User: test@example.com / testpass123

Code Architecture

Core Application Structure

main.py: FastAPI application with:

• JWT cookie-based authentication system
• HTML routes for web interface
• REST API endpoints for agent/user management
• Three-tier role-based access control (admin / readonly_admin / user)
• Client verification workflow with email notifications
• Daily agent digest scheduler (APScheduler)
• Template rendering with Jinja2

Key Authentication Functions:

• get_current_user_optional(): Cookie-based auth for templates
• get_current_user_from_cookie(): Required auth for API endpoints
• require_admin(): Admin-only access control (write operations)
• require_admin_or_readonly(): Admin + readonly_admin access (read-only dashboard views)

Data Layer

models.py: Pydantic models for:

• AiAgent: Core agent model with comprehensive fields (includes discipline, rating, client, client_name, studio_name)
• UsageTimelineEntry: Daily usage data including message_count and token_count
• UserCreate/UserResponse: User management models (includes role field: user/admin/readonly_admin)
• UserUpdate: Includes role field for three-tier role management
• AiAgentCreate/AiAgentResponse: API request/response models (includes total_tokens, prompt_tokens, completion_tokens, discipline, rating, rating_count, client, client_name, studio_name, verification_status, verified_by, verified_date)
• AgentCollectorCreate: Collector API input model (includes total_tokens, prompt_tokens, completion_tokens, discipline, client, client_name, studio_name)
• AgentUsageStatsResponse: Usage statistics response (includes total_tokens, prompt_tokens, completion_tokens)

crud.py: Database operations using Motor (async MongoDB driver):

• User CRUD: authentication, creation, management
• Agent CRUD: create, read, update, delete with user ownership
• Advanced features: search, filtering, statistics, pagination
• All operations use ObjectId for MongoDB document IDs

database.py: MongoDB connection setup with Motor async client

• Collections: users, agents, agent_usage, token_notifications, agent_ratings
• ensure_indexes(): Creates compound unique index on agent_ratings(agent_id, user_id) and index on verification_status

notifications.py: Mailgun email notification system:

• is_mailgun_configured(): Returns False if env vars not set (gracefully disabled)
• send_mailgun_email(): POST to Mailgun HTTP API with 10s timeout
• build_threshold_email(): HTML email template for threshold alerts
• check_and_notify_threshold(): Checks token usage against threshold, enforces cooldown via token_notifications collection, sends to admin users
• send_client_agent_notification(): Sends email when client-facing agent is created (to CLIENT_AGENT_NOTIFY_EMAILS)
• build_client_agent_email(): HTML email template for client agent notifications
• send_daily_agent_digest(): Queries agents created in last 24 hours, sends summary to all admin users
• build_daily_digest_email(): HTML email template for daily digest

auth.py: JWT authentication with:

• bcrypt password hashing
• JWT token creation/validation using python-jose
• Configurable token expiration

Frontend Templates

Located in templates/ directory:

• base.html: Bootstrap 5 base template with navigation
• nav.html: Dynamic navigation based on user role
• index.html: Landing page
• login.html/register.html: Authentication forms
• agent_register.html: Agent creation form
• agent_management.html: Agent dashboard with real data
• search.html: Global search functionality
• user_management.html: User management interface
• admin/dashboard.html: Admin statistics, management, and verification workflow

Static Assets

static/style.css: Custom CSS with:

• CSS variables for consistent theming
• Gradient backgrounds and modern styling
• Responsive design for mobile devices
• Bootstrap 5 customizations

Key Features

Authentication Flow

• Cookie-based JWT authentication
• Three-tier role-based access: user, admin, readonly_admin
  • user: Standard access, can manage own agents
  • admin: Full access to admin dashboard, all write operations
  • readonly_admin: Can view admin dashboard but all write actions (edit, delete, approve, create) are hidden
• role field on user documents; is_admin kept in sync for backward compatibility
• require_admin_or_readonly() dependency for read-only admin endpoints
• Automatic redirects based on user role (admin/readonly_admin → /admin, user → /agent-management)
• Secure logout with token cleanup

Agent Management

• Full CRUD operations with user ownership
• Status tracking (Active, Inactive, Development, Deprecated)
• Rich metadata: tags, userbase, department, contact person
• Search functionality across multiple fields
• Filtering by status, audit status (Audited / Not Audited), and discipline
• Admin can view/manage all agents

Client & Verification System

• client field on agents: "yes" or "no" (mandatory on registration form)
• client_name: free text, required when client is "yes"
• studio_name: optional free text field
• Registration form order: Name, Description, Purpose, Client (Yes/No), Client Name (conditional), Studio Name, Tool, then Version/Status/etc.
• When client == "yes": agent auto-tagged with verification_status = "needs_verification"
• Verification tab on admin dashboard shows pending agents with Approve button
• PUT /api/admin/agents/{id}/verify — admin-only, sets status to "verified" with verifier info
• GET /api/admin/agents/pending-verification — returns agents needing verification
• Verification badges displayed on agent cards (orange "Needs Verification", green "Verified")

Client Agent Email Notification

• When client == "yes" on agent creation, sends email via Mailgun to CLIENT_AGENT_NOTIFY_EMAILS
• Subject: "Client Agent Created"
• Body includes: Agent Name, Description, Purpose, Client Name, Studio Name, Tool, Created By
• Non-blocking: failure does not break agent creation

Daily Agent Digest Email

• Scheduled via APScheduler at configured hour (default 7:00 AM, DAILY_DIGEST_HOUR env var)
• Queries agents created in last 24 hours
• Sends to all active admin users
• Body includes: Agent Name, Purpose, Description, Created By (email)
• Subject: "Agents Created Last 24 Hours"
• Skips sending if no agents created
• Can be manually triggered via POST /api/admin/digest/send (admin-only)

Discipline & Star Rating

• discipline field classifies agents into business categories: Strategy, Creative, Oversight including delivery, Optimization, Back Office including operations, Pencil Agents
• Required on registration form, optional on edit (to support legacy agents)
• Pencil Agents discipline is auto-assigned to agents with "pencil" in the name when no discipline is set (collector API auto-tag + startup migration)
• rating field stores the average star rating (1-5) computed from all per-user ratings
• rating_count field stores the number of individual ratings
• Per-user rating system: Any authenticated user can rate any agent via PUT /api/agents/{id}/rating
  • Individual ratings stored in agent_ratings collection with compound unique index on (agent_id, user_id)
  • After each rating, the agent's average rating and count are recalculated and stored on the agent document
  • GET /api/agents/{id}/my-rating returns the current user's rating plus the average and count
• Interactive star rating widget in detail modal shows the user's own rating as filled stars
• Average rating and count displayed below the stars and on agent card badges
• Rating removed from edit modals (rating is per-user, not admin-set)
• Rating framework info modal accessible via info icon next to "Rating:" label
• Dashboard supports filtering by discipline and sorting by rating
• Discipline badge (purple) and star rating badge displayed on agent cards
• Both fields included in CSV export/import
• Discipline passed through collector API; rating is human-only (not in collector)

Token Usage Tracking

• total_tokens, prompt_tokens, completion_tokens fields on agents track cumulative LLM token consumption
• prompt_tokens (input) and completion_tokens (output) provide cost breakdown detail
• token_count per day in usage timeline entries alongside message_count
• Token badge displayed on agent cards (gold/coins icon) with prompt/completion breakdown in tooltip
• Usage modal shows Total Tokens stat with In/Out breakdown alongside messages/conversations/users
• Dual-axis chart (messages left axis, tokens right axis) when token data exists
• Sort agents by Total Tokens
• CSV export includes total_tokens, prompt_tokens, completion_tokens columns

High Usage Email Notifications

• Entirely optional — silently disabled when Mailgun env vars are not set
• Triggered from the Agent Collector endpoint (POST /agents) when total_tokens exceeds threshold
• Non-blocking — notification failure never breaks the collector API
• Cooldown tracking in MongoDB token_notifications collection (default 24h, configurable)
• Sends to all active admin users' email addresses

User Management

• User registration with validation
• Admin user creation capabilities
• Three-tier role system: user, admin, readonly_admin
  • Role dropdown in admin user edit modal (replaces is_admin checkbox)
  • role and is_admin fields kept in sync on update
• Profile management
• User statistics and administration

Database Integration

• MongoDB with proper ObjectId handling
• Async operations using Motor driver
• Indexed queries for performance
• Data aggregation for statistics
• Collections: users, agents, agent_usage, token_notifications, agent_ratings

Development Guidelines

Database Operations

• Always use ObjectId for MongoDB document IDs
• Use Motor async driver methods (await collection.find_one())
• Handle ObjectId conversion in CRUD operations
• Implement proper error handling with try/except blocks

Authentication

• Use cookie-based auth for web interface
• API endpoints require get_current_user_from_cookie() dependency
• Write endpoints use require_admin() dependency
• Read-only admin endpoints use require_admin_or_readonly() dependency
• Always validate user permissions for data access
• readonly_admin users can view admin dashboard but UI hides all write-action buttons via admin-write-action CSS class

Template Context

• Pass current_user to all templates for navigation
• Handle dict objects (not User model instances) in templates
• Use proper null checks for optional user data

API Response Models

• Convert ObjectId to string in API responses
• Handle optional datetime fields with isoformat()
• Maintain consistency between Create and Response models

Error Handling

• Provide meaningful error messages in templates
• Use proper HTTP status codes in API responses
• Graceful degradation for missing data

Project Dependencies

Key dependencies from requirements.txt:

• fastapi: Web framework
• uvicorn: ASGI server
• motor: Async MongoDB driver
• pymongo: MongoDB operations
• python-jose: JWT token handling
• passlib: Password hashing
• bcrypt: Password encryption
• pydantic: Data validation
• jinja2: Template engine
• python-multipart: Form handling
• requests: HTTP client (used for Mailgun API calls)
• apscheduler: Task scheduling (daily digest email)

API Endpoints (New)

Verification

• GET /api/admin/agents/pending-verification — List agents with verification status (admin + readonly_admin)
• PUT /api/admin/agents/{agent_id}/verify — Approve/verify an agent (admin only)

Daily Digest

• POST /api/admin/digest/send — Manually trigger the daily agent digest email (admin only)