Oliver/agent_tracker
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add per-user rating system, rating framework modal, and Pencil Agents discipline

Browse source 
- Replace single-owner rating with per-user ratings stored in agent_ratings collection
- Any authenticated user can now rate any agent; average is computed and stored on agent doc
- Add GET /api/agents/{id}/my-rating endpoint for fetching user's own rating
- Add rating framework info modal showing 1-5 scale definitions and 4 performance areas
- Add "Pencil Agents" discipline with auto-tagging for agents with "pencil" in name
- Run Pencil migration on startup and auto-tag in collector API
- Remove star rating from edit modals (rating is now per-user, not admin-set)
- Add rating_count field to AiAgentResponse and CSV export

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
nickviljoen 2026-02-24 08:34:41 +02:00
parent a50ef3ec64
commit 62138e9142
8 changed files with 327 additions and 160 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

23
CLAUDE.md
View file

@ -60,7 +60,7 @@ Create `.env` file with:
- `AiAgent`: Core agent model with comprehensive fields (includes `discipline`, `rating`)
- `UsageTimelineEntry`: Daily usage data including `message_count` and `token_count`
- `UserCreate/UserResponse`: User management models
- `AiAgentCreate/AiAgentResponse`: API request/response models (includes `total_tokens`, `discipline`, `rating`)
- `AiAgentCreate/AiAgentResponse`: API request/response models (includes `total_tokens`, `discipline`, `rating`, `rating_count`)
- `AgentCollectorCreate`: Collector API input model (includes `total_tokens`, `discipline`)
- `AgentUsageStatsResponse`: Usage statistics response (includes `total_tokens`)
@ -71,7 +71,8 @@ Create `.env` file with:
- All operations use ObjectId for MongoDB document IDs
**database.py**: MongoDB connection setup with Motor async client
- Collections: `users`, `agents`, `agent_usage`, `token_notifications`
- Collections: `users`, `agents`, `agent_usage`, `token_notifications`, `agent_ratings`
- `ensure_indexes()`: Creates compound unique index on `agent_ratings(agent_id, user_id)`
**notifications.py**: Optional Mailgun email notification system:
- `is_mailgun_configured()`: Returns False if env vars not set (gracefully disabled)
@ -122,11 +123,19 @@ Located in `templates/` directory:
- Admin can view/manage all agents
### Discipline & Star Rating
- `discipline` field classifies agents into business categories: Strategy, Creative, Oversight including delivery, Optimization, Back Office including operations
- `discipline` field classifies agents into business categories: Strategy, Creative, Oversight including delivery, Optimization, Back Office including operations, Pencil Agents
- Required on registration form, optional on edit (to support legacy agents)
- `rating` field is a 1-5 star rating for quality assessment by discipline leads
- Interactive star rating widget in detail modal saves immediately via `PUT /api/agents/{id}/rating`
- Star rating also editable in the edit modal (both agent management page and admin dashboard)
- Pencil Agents discipline is auto-assigned to agents with "pencil" in the name when no discipline is set (collector API auto-tag + startup migration)
- `rating` field stores the **average** star rating (1-5) computed from all per-user ratings
- `rating_count` field stores the number of individual ratings
- **Per-user rating system**: Any authenticated user can rate any agent via `PUT /api/agents/{id}/rating`
- Individual ratings stored in `agent_ratings` collection with compound unique index on `(agent_id, user_id)`
- After each rating, the agent's average rating and count are recalculated and stored on the agent document
- `GET /api/agents/{id}/my-rating` returns the current user's rating plus the average and count
- Interactive star rating widget in detail modal shows the **user's own rating** as filled stars
- Average rating and count displayed below the stars and on agent card badges
- Rating removed from edit modals (rating is per-user, not admin-set)
- Rating framework info modal accessible via info icon next to "Rating:" label
- Dashboard supports filtering by discipline and sorting by rating
- Discipline badge (purple) and star rating badge displayed on agent cards
- Both fields included in CSV export/import
@ -159,7 +168,7 @@ Located in `templates/` directory:
- Async operations using Motor driver
- Indexed queries for performance
- Data aggregation for statistics
- Collections: `users`, `agents`, `agent_usage`, `token_notifications`
- Collections: `users`, `agents`, `agent_usage`, `token_notifications`, `agent_ratings`
## Development Guidelines

72
crud.py
View file

@ -1,7 +1,7 @@
from datetime import datetime, timedelta
from bson import ObjectId
import database
from database import users_collection, agents_collection, agent_usage_collection
from database import users_collection, agents_collection, agent_usage_collection, agent_ratings_collection
import auth
from auth import hash_password, verify_password
import re
@ -539,3 +539,73 @@ async def create_agent_from_collector(agent_data: dict):
return agent_doc
except Exception as e:
raise Exception(f"Failed to store agent data: {str(e)}")
# Per-user rating functions
async def upsert_agent_rating(agent_id: str, user_id: str, rating: float):
"""Insert or update a user's individual rating for an agent"""
now = datetime.utcnow()
result = await agent_ratings_collection.update_one(
{"agent_id": agent_id, "user_id": user_id},
{
"$set": {"rating": rating, "updated_at": now},
"$setOnInsert": {"created_at": now}
},
upsert=True
)
return result
async def get_agent_rating_stats(agent_id: str):
"""Get average rating and count for an agent via aggregation"""
pipeline = [
{"$match": {"agent_id": agent_id}},
{"$group": {
"_id": None,
"avg_rating": {"$avg": "$rating"},
"count": {"$sum": 1}
}}
]
results = await agent_ratings_collection.aggregate(pipeline).to_list(length=1)
if results:
return {
"avg_rating": round(results[0]["avg_rating"], 1),
"count": results[0]["count"]
}
return {"avg_rating": None, "count": 0}
async def get_user_rating_for_agent(agent_id: str, user_id: str):
"""Fetch current user's own rating for an agent"""
doc = await agent_ratings_collection.find_one(
{"agent_id": agent_id, "user_id": user_id}
)
return doc["rating"] if doc else None
async def update_agent_average_rating(agent_id: str):
"""Recalculate and store average rating on the agent document"""
stats = await get_agent_rating_stats(agent_id)
update_data = {
"rating": stats["avg_rating"],
"rating_count": stats["count"],
"updated_at": datetime.utcnow()
}
await agents_collection.update_one(
{"_id": ObjectId(agent_id)},
{"$set": update_data}
)
return stats
async def migrate_pencil_agents_discipline():
"""Update agents with 'pencil' in name that have no discipline set to 'Pencil Agents'"""
result = await agents_collection.update_many(
{
"agent_name": {"$regex": "pencil", "$options": "i"},
"$or": [
{"discipline": None},
{"discipline": ""},
{"discipline": {"$exists": False}}
]
},
{"$set": {"discipline": "Pencil Agents"}}
)
return result.modified_count

12
database.py
View file

@ -13,6 +13,18 @@ users_collection = db.get_collection("users")
agents_collection = db.get_collection("agents")
agent_usage_collection = db.get_collection("agent_usage")
notifications_collection = db.get_collection("token_notifications")
agent_ratings_collection = db.get_collection("agent_ratings")
async def ensure_indexes():
"""Create database indexes for performance"""
try:
await agent_ratings_collection.create_index(
[("agent_id", 1), ("user_id", 1)],
unique=True
)
print("Database indexes ensured successfully")
except Exception as e:
print(f"Warning: Failed to create indexes: {e}")
async def check_database_health():
"""Check MongoDB connection health"""

60
main.py
View file

@ -227,6 +227,7 @@ def create_agent_response(agent: dict) -> models.AiAgentResponse:
last_edited_by=agent.get("last_edited_by"),
discipline=agent.get("discipline"),
rating=agent.get("rating"),
rating_count=agent.get("rating_count"),
created_by=agent["created_by"],
# Usage tracking fields
usage_timeline=agent.get("usage_timeline"),
@ -266,6 +267,11 @@ def map_agent_collector_to_internal(collector_data: models.AgentCollectorCreate)
usage_timeline = [{"date": entry.date, "message_count": entry.message_count, "token_count": entry.token_count}
for entry in collector_data.usage_timeline]
# Auto-tag Pencil Agents discipline
discipline = collector_data.discipline
if not discipline and "pencil" in collector_data.name.lower():
discipline = "Pencil Agents"
return {
"agent_name": collector_data.name,
"agent_tool": collector_data.tool,
@ -281,7 +287,7 @@ def map_agent_collector_to_internal(collector_data: models.AgentCollectorCreate)
"agent_tags": collector_data.tags,
"agent_metadata": collector_data.metadata,
"url": collector_data.url,
"discipline": collector_data.discipline,
"discipline": discipline,
# Usage tracking fields
"usage_timeline": usage_timeline,
"conversation_count": collector_data.conversation_count,
@ -303,6 +309,17 @@ async def me(current_user: dict = Depends(get_current_user)):
"is_admin": current_user["is_admin"]
}
@app.on_event("startup")
async def startup_event():
"""Run database migrations and ensure indexes on startup"""
from database import ensure_indexes
await ensure_indexes()
# Migrate pencil agents discipline
count = await crud.migrate_pencil_agents_discipline()
if count > 0:
print(f"Pencil Agents migration: updated {count} agent(s)")
# HTML Routes
@app.get("/")
async def home(request: Request):
@ -893,25 +910,44 @@ async def update_agent(agent_id: str, agent: models.AiAgentCreate, current_user:
@app.put("/api/agents/{agent_id}/rating")
async def update_agent_rating(agent_id: str, request: Request, current_user: dict = Depends(get_current_user_from_cookie)):
"""Update only the star rating of an agent (lightweight endpoint for immediate saves)"""
"""Submit a per-user star rating for an agent (any authenticated user can rate)"""
existing_agent = await crud.get_agent_by_id(agent_id)
if not existing_agent:
raise HTTPException(status_code=404, detail="Agent not found")
if not can_user_edit_agent(existing_agent, current_user):
raise HTTPException(status_code=403, detail="Not authorized to rate this agent")
body = await request.json()
rating_value = body.get("rating")
if rating_value is None or not (1 <= float(rating_value) <= 5):
raise HTTPException(status_code=422, detail="Rating must be between 1 and 5")
updated = await crud.update_agent(agent_id, {"rating": float(rating_value)}, last_edited_by=current_user.get("email"))
if not updated:
raise HTTPException(status_code=500, detail="Failed to update rating")
user_id = str(current_user["_id"])
await crud.upsert_agent_rating(agent_id, user_id, float(rating_value))
stats = await crud.update_agent_average_rating(agent_id)
return {"message": "Rating updated", "rating": float(rating_value)}
return {
"message": "Rating updated",
"rating": stats["avg_rating"],
"rating_count": stats["count"],
"user_rating": float(rating_value)
}
@app.get("/api/agents/{agent_id}/my-rating")
async def get_my_agent_rating(agent_id: str, current_user: dict = Depends(get_current_user_from_cookie)):
"""Get the current user's rating for an agent plus the average"""
existing_agent = await crud.get_agent_by_id(agent_id)
if not existing_agent:
raise HTTPException(status_code=404, detail="Agent not found")
user_id = str(current_user["_id"])
user_rating = await crud.get_user_rating_for_agent(agent_id, user_id)
stats = await crud.get_agent_rating_stats(agent_id)
return {
"user_rating": user_rating,
"rating": stats["avg_rating"],
"rating_count": stats["count"]
}
@app.delete("/api/agents/{agent_id}")
async def delete_agent(agent_id: str, current_user: dict = Depends(get_current_user_from_cookie)):
@ -1115,7 +1151,8 @@ async def export_agents_csv(current_user: dict = Depends(require_admin)):
"created_by",
"total_tokens",
"discipline",
"rating"
"rating",
"rating_count"
]
writer = csv.DictWriter(output, fieldnames=fieldnames)
@ -1152,7 +1189,8 @@ async def export_agents_csv(current_user: dict = Depends(require_admin)):
"created_by": agent.get("created_by", ""),
"total_tokens": str(agent.get("total_tokens", "")) if agent.get("total_tokens") is not None else "",
"discipline": agent.get("discipline", ""),
"rating": str(agent.get("rating", "")) if agent.get("rating") is not None else ""
"rating": str(agent.get("rating", "")) if agent.get("rating") is not None else "",
"rating_count": str(agent.get("rating_count", "")) if agent.get("rating_count") is not None else ""
}
writer.writerow(row)

1
models.py
View file

@ -129,6 +129,7 @@ class AiAgentResponse(BaseModel):
last_edited_by: Optional[str] = None
discipline: Optional[str] = None
rating: Optional[float] = None
rating_count: Optional[int] = None
created_by: str
# Usage tracking fields (new)

136
templates/admin/dashboard.html
View file

@ -429,21 +429,9 @@
<option value="Oversight including delivery">Oversight including delivery</option>
<option value="Optimization">Optimization</option>
<option value="Back Office including operations">Back Office including operations</option>
<option value="Pencil Agents">Pencil Agents</option>
</select>
</div>
<div class="col-md-6 mb-3">
<label class="form-label">
<i class="fas fa-star me-2"></i>Rating
</label>
<div class="star-rating-edit" id="editStarRating">
<span class="star-interactive" data-value="1">&#9733;</span>
<span class="star-interactive" data-value="2">&#9733;</span>
<span class="star-interactive" data-value="3">&#9733;</span>
<span class="star-interactive" data-value="4">&#9733;</span>
<span class="star-interactive" data-value="5">&#9733;</span>
</div>
<input type="hidden" id="editAgentRating" value="">
</div>
</div>
<div class="mb-3" id="editQualityAuditSection">
@ -487,6 +475,71 @@
</div>
</div>
<!-- Rating Framework Info Modal -->
<div class="modal fade" id="ratingFrameworkModal" tabindex="-1">
<div class="modal-dialog modal-lg">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title"><i class="fas fa-info-circle me-2"></i>Rating Framework</h5>
<button type="button" class="btn-close" data-bs-dismiss="modal"></button>
</div>
<div class="modal-body">
<h6 class="mb-3">Star Rating Scale</h6>
<table class="table table-bordered table-sm">
<thead class="table-light">
<tr><th>Rating</th><th>Level</th><th>Description</th></tr>
</thead>
<tbody>
<tr><td><span class="text-warning">&#9733;</span> 1</td><td>Initial</td><td>Agent is in early stages; minimal functionality, limited testing, and not yet ready for regular use.</td></tr>
<tr><td><span class="text-warning">&#9733;&#9733;</span> 2</td><td>Developing</td><td>Agent has basic functionality but significant gaps remain; occasional errors and inconsistent outputs.</td></tr>
<tr><td><span class="text-warning">&#9733;&#9733;&#9733;</span> 3</td><td>Competent</td><td>Agent performs core tasks reliably; meets basic expectations with acceptable accuracy and response quality.</td></tr>
<tr><td><span class="text-warning">&#9733;&#9733;&#9733;&#9733;</span> 4</td><td>Proficient</td><td>Agent delivers high-quality results consistently; handles edge cases well and provides good user experience.</td></tr>
<tr><td><span class="text-warning">&#9733;&#9733;&#9733;&#9733;&#9733;</span> 5</td><td>Expert</td><td>Agent excels across all dimensions; exceptional accuracy, reliability, and user satisfaction.</td></tr>
</tbody>
</table>
<h6 class="mt-4 mb-3">Performance Areas</h6>
<div class="row">
<div class="col-md-6 mb-3">
<div class="card h-100">
<div class="card-body">
<h6 class="card-title"><i class="fas fa-bullseye me-2 text-primary"></i>Accuracy & Reliability</h6>
<p class="card-text small text-muted mb-0">How correct and consistent are the agent's outputs? Does it handle inputs reliably without errors?</p>
</div>
</div>
</div>
<div class="col-md-6 mb-3">
<div class="card h-100">
<div class="card-body">
<h6 class="card-title"><i class="fas fa-user-check me-2 text-success"></i>User Experience</h6>
<p class="card-text small text-muted mb-0">Is the agent easy to interact with? Does it understand user intent and provide clear, helpful responses?</p>
</div>
</div>
</div>
<div class="col-md-6 mb-3">
<div class="card h-100">
<div class="card-body">
<h6 class="card-title"><i class="fas fa-tachometer-alt me-2 text-warning"></i>Efficiency & Performance</h6>
<p class="card-text small text-muted mb-0">Does the agent respond promptly? Does it complete tasks without unnecessary steps or resource usage?</p>
</div>
</div>
</div>
<div class="col-md-6 mb-3">
<div class="card h-100">
<div class="card-body">
<h6 class="card-title"><i class="fas fa-expand-arrows-alt me-2 text-info"></i>Scope & Adaptability</h6>
<p class="card-text small text-muted mb-0">How well does the agent handle varied or unexpected scenarios? Can it adapt to different user needs?</p>
</div>
</div>
</div>
</div>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-secondary" data-bs-dismiss="modal">Close</button>
</div>
</div>
</div>
</div>
<style>
.stat-card {
background: linear-gradient(135deg, #f3ae3e 0%, #f3ae3e 100%);
@ -666,21 +719,6 @@
color: white !important;
}
.star-interactive {
cursor: pointer;
font-size: 1.3rem;
transition: color 0.2s, transform 0.2s;
color: #dee2e6;
}
.star-interactive:hover {
transform: scale(1.2);
}
.star-rating-edit {
padding: 8px 0;
}
@media (max-width: 768px) {
.stat-card {
text-align: center;
@ -705,7 +743,6 @@ let allAgents = [];
document.addEventListener('DOMContentLoaded', function() {
loadAdminData();
setupEventListeners();
setupEditStarRating();
});
function setupEventListeners() {
@ -1164,8 +1201,6 @@ async function editAgentAdmin(agentId) {
document.getElementById('editAgentUserbase').value = agent.agent_userbase ? agent.agent_userbase.join(', ') : '';
document.getElementById('editAgentCapabilities').value = agent.agent_capabilities ? agent.agent_capabilities.join(', ') : '';
document.getElementById('editAgentDiscipline').value = agent.discipline || '';
document.getElementById('editAgentRating').value = agent.rating || '';
updateEditStarDisplay(agent.rating || 0);
// Handle Quality Audit field (admin always has access)
document.getElementById('editQualityAuditStatus').checked = agent.quality_audit_status || false;
@ -1222,7 +1257,6 @@ async function handleEditAgentSubmit(e) {
agent_userbase: document.getElementById('editAgentUserbase').value.split(',').map(s => s.trim()).filter(s => s),
agent_capabilities: document.getElementById('editAgentCapabilities').value.split(',').map(s => s.trim()).filter(s => s),
discipline: document.getElementById('editAgentDiscipline').value || null,
rating: document.getElementById('editAgentRating').value ? parseFloat(document.getElementById('editAgentRating').value) : null,
quality_audit_status: document.getElementById('editQualityAuditStatus').checked
};
@ -1397,44 +1431,6 @@ async function handleResetPasswordSubmit(e) {
}
}
// Star rating functions for edit modal
function setupEditStarRating() {
const container = document.getElementById('editStarRating');
if (!container) return;
const stars = container.querySelectorAll('.star-interactive');
const hiddenInput = document.getElementById('editAgentRating');
stars.forEach(star => {
star.addEventListener('mouseenter', function() {
const val = parseInt(this.dataset.value);
stars.forEach(s => {
s.style.color = parseInt(s.dataset.value) <= val ? '#ffc107' : '#dee2e6';
});
});
star.addEventListener('mouseleave', function() {
const current = parseInt(hiddenInput.value) || 0;
stars.forEach(s => {
s.style.color = parseInt(s.dataset.value) <= current ? '#ffc107' : '#dee2e6';
});
});
star.addEventListener('click', function() {
const val = parseInt(this.dataset.value);
hiddenInput.value = val;
stars.forEach(s => {
s.style.color = parseInt(s.dataset.value) <= val ? '#ffc107' : '#dee2e6';
});
});
});
}
function updateEditStarDisplay(rating) {
const container = document.getElementById('editStarRating');
if (!container) return;
const stars = container.querySelectorAll('.star-interactive');
const val = Math.round(rating) || 0;
stars.forEach(s => {
s.style.color = parseInt(s.dataset.value) <= val ? '#ffc107' : '#dee2e6';
});
}
</script>
{% endblock %}

182
templates/agent_management.html
View file

@ -107,6 +107,7 @@
<option value="Oversight including delivery">Oversight including delivery</option>
<option value="Optimization">Optimization</option>
<option value="Back Office including operations">Back Office including operations</option>
<option value="Pencil Agents">Pencil Agents</option>
</select>
</div>
<div class="col-md-2 mb-3 mb-md-0">
@ -340,21 +341,9 @@
<option value="Oversight including delivery">Oversight including delivery</option>
<option value="Optimization">Optimization</option>
<option value="Back Office including operations">Back Office including operations</option>
<option value="Pencil Agents">Pencil Agents</option>
</select>
</div>
<div class="col-md-6 mb-3">
<label class="form-label">
<i class="fas fa-star me-2"></i>Rating
</label>
<div class="star-rating-edit" id="editStarRating">
<span class="star-interactive" data-value="1">&#9733;</span>
<span class="star-interactive" data-value="2">&#9733;</span>
<span class="star-interactive" data-value="3">&#9733;</span>
<span class="star-interactive" data-value="4">&#9733;</span>
<span class="star-interactive" data-value="5">&#9733;</span>
</div>
<input type="hidden" id="editAgentRating" value="">
</div>
</div>
<div class="mb-3" id="editQualityAuditSection">
@ -398,6 +387,71 @@
</div>
</div>
<!-- Rating Framework Info Modal -->
<div class="modal fade" id="ratingFrameworkModal" tabindex="-1">
<div class="modal-dialog modal-lg">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title"><i class="fas fa-info-circle me-2"></i>Rating Framework</h5>
<button type="button" class="btn-close" data-bs-dismiss="modal"></button>
</div>
<div class="modal-body">
<h6 class="mb-3">Star Rating Scale</h6>
<table class="table table-bordered table-sm">
<thead class="table-light">
<tr><th>Rating</th><th>Level</th><th>Description</th></tr>
</thead>
<tbody>
<tr><td><span class="text-warning">&#9733;</span> 1</td><td>Initial</td><td>Agent is in early stages; minimal functionality, limited testing, and not yet ready for regular use.</td></tr>
<tr><td><span class="text-warning">&#9733;&#9733;</span> 2</td><td>Developing</td><td>Agent has basic functionality but significant gaps remain; occasional errors and inconsistent outputs.</td></tr>
<tr><td><span class="text-warning">&#9733;&#9733;&#9733;</span> 3</td><td>Competent</td><td>Agent performs core tasks reliably; meets basic expectations with acceptable accuracy and response quality.</td></tr>
<tr><td><span class="text-warning">&#9733;&#9733;&#9733;&#9733;</span> 4</td><td>Proficient</td><td>Agent delivers high-quality results consistently; handles edge cases well and provides good user experience.</td></tr>
<tr><td><span class="text-warning">&#9733;&#9733;&#9733;&#9733;&#9733;</span> 5</td><td>Expert</td><td>Agent excels across all dimensions; exceptional accuracy, reliability, and user satisfaction.</td></tr>
</tbody>
</table>
<h6 class="mt-4 mb-3">Performance Areas</h6>
<div class="row">
<div class="col-md-6 mb-3">
<div class="card h-100">
<div class="card-body">
<h6 class="card-title"><i class="fas fa-bullseye me-2 text-primary"></i>Accuracy & Reliability</h6>
<p class="card-text small text-muted mb-0">How correct and consistent are the agent's outputs? Does it handle inputs reliably without errors?</p>
</div>
</div>
</div>
<div class="col-md-6 mb-3">
<div class="card h-100">
<div class="card-body">
<h6 class="card-title"><i class="fas fa-user-check me-2 text-success"></i>User Experience</h6>
<p class="card-text small text-muted mb-0">Is the agent easy to interact with? Does it understand user intent and provide clear, helpful responses?</p>
</div>
</div>
</div>
<div class="col-md-6 mb-3">
<div class="card h-100">
<div class="card-body">
<h6 class="card-title"><i class="fas fa-tachometer-alt me-2 text-warning"></i>Efficiency & Performance</h6>
<p class="card-text small text-muted mb-0">Does the agent respond promptly? Does it complete tasks without unnecessary steps or resource usage?</p>
</div>
</div>
</div>
<div class="col-md-6 mb-3">
<div class="card h-100">
<div class="card-body">
<h6 class="card-title"><i class="fas fa-expand-arrows-alt me-2 text-info"></i>Scope & Adaptability</h6>
<p class="card-text small text-muted mb-0">How well does the agent handle varied or unexpected scenarios? Can it adapt to different user needs?</p>
</div>
</div>
</div>
</div>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-secondary" data-bs-dismiss="modal">Close</button>
</div>
</div>
</div>
</div>
<style>
.agent-card {
border: 1px solid #e9ecef;
@ -502,7 +556,6 @@ document.addEventListener('DOMContentLoaded', function() {
loadAgentsForCurrentView();
});
setupEventListeners();
setupEditStarRating();
});
function setupEventListeners() {
@ -665,7 +718,7 @@ function displayAgents(agentsToShow) {
</span>
${agent.agent_version ? `<span class="badge bg-light text-dark">v${agent.agent_version}</span>` : ''}
${agent.discipline ? `<span class="badge bg-purple" title="Discipline"><i class="fas fa-layer-group me-1"></i>${agent.discipline}</span>` : ''}
${agent.rating ? `<span class="badge bg-warning text-dark" title="Rating: ${agent.rating}/5"><i class="fas fa-star me-1"></i>${agent.rating.toFixed(1)}</span>` : ''}
${agent.rating ? `<span class="badge bg-warning text-dark" title="Rating: ${agent.rating}/5"><i class="fas fa-star me-1"></i>${agent.rating.toFixed(1)}${agent.rating_count ? ' (' + agent.rating_count + ')' : ''}</span>` : ''}
${agent.quality_audit_status ? '<span class="badge bg-success" title="Quality Audited"><i class="fas fa-certificate"></i></span>' : ''}
${agent.quality_audit_status && agent.risk_factor ? getRiskFactorBadge(agent.risk_factor) : ''}
${agent.total_messages ? `<span class="badge bg-info text-white" title="Total Messages"><i class="fas fa-comments me-1"></i>${agent.total_messages.toLocaleString()}</span>` : ''}
@ -755,8 +808,8 @@ async function showAgentDetails(agentId) {
<tr><td><strong>Purpose:</strong></td><td>${agent.agent_purpose || 'N/A'}</td></tr>
${agent.url ? `<tr><td><strong>Agent Link:</strong></td><td><a href="${agent.url}" target="_blank" class="btn btn-sm btn-success"><i class="fas fa-external-link-alt me-1"></i>Open Agent</a></td></tr>` : ''}
<tr><td><strong>Discipline:</strong></td><td>${agent.discipline ? `<span class="badge bg-purple"><i class="fas fa-layer-group me-1"></i>${agent.discipline}</span>` : '<span class="text-muted">N/A</span>'}</td></tr>
<tr><td><strong>Rating:</strong></td><td>
<span id="detailRatingDisplay" data-agent-id="${agent.agent_id}">${agent.rating ? getStarDisplay(agent.rating) : '<span class="text-muted">Not yet rated</span>'}</span>
<tr><td><strong>Rating:</strong> <a href="#" onclick="event.preventDefault(); new bootstrap.Modal(document.getElementById('ratingFrameworkModal')).show();" title="View rating framework"><i class="fas fa-info-circle text-primary"></i></a></td><td>
<span id="detailRatingDisplay" data-agent-id="${agent.agent_id}">${agent.rating ? getStarDisplay(agent.rating, agent.rating_count) : '<span class="text-muted">Not yet rated</span>'}</span>
</td></tr>
<tr><td><strong>Quality Audit:</strong></td><td>
${agent.quality_audit_status ?
@ -901,13 +954,31 @@ async function showAgentDetails(agentId) {
const modal = new bootstrap.Modal(document.getElementById('agentModal'));
modal.show();
// Add clickable star rating if user can edit
if (canEdit) {
// All authenticated users can rate - fetch user's own rating and show interactive stars
{
const ratingContainer = document.getElementById('detailRatingDisplay');
if (ratingContainer) {
const currentRating = agent.rating || 0;
ratingContainer.innerHTML = getClickableStars(currentRating, agent.agent_id);
setupDetailStarListeners(agent.agent_id, currentRating);
try {
const ratingResp = await fetch(`{{ base_path }}/api/agents/${agent.agent_id}/my-rating`, { credentials: 'include' });
if (ratingResp.ok) {
const ratingData = await ratingResp.json();
const userRating = ratingData.user_rating || 0;
ratingContainer.innerHTML = getClickableStars(userRating, agent.agent_id);
// Show average below the stars
if (ratingData.rating) {
ratingContainer.innerHTML += ` <small class="text-muted">Avg: ${ratingData.rating.toFixed(1)} (${ratingData.rating_count} rating${ratingData.rating_count !== 1 ? 's' : ''})</small>`;
}
setupDetailStarListeners(agent.agent_id, userRating);
} else {
const currentRating = agent.rating || 0;
ratingContainer.innerHTML = getClickableStars(currentRating, agent.agent_id);
setupDetailStarListeners(agent.agent_id, currentRating);
}
} catch (err) {
const currentRating = agent.rating || 0;
ratingContainer.innerHTML = getClickableStars(currentRating, agent.agent_id);
setupDetailStarListeners(agent.agent_id, currentRating);
}
}
}
@ -964,8 +1035,6 @@ async function showEditModal() {
document.getElementById('editAgentUserbase').value = agent.agent_userbase ? agent.agent_userbase.join(', ') : '';
document.getElementById('editAgentCapabilities').value = agent.agent_capabilities ? agent.agent_capabilities.join(', ') : '';
document.getElementById('editAgentDiscipline').value = agent.discipline || '';
document.getElementById('editAgentRating').value = agent.rating || '';
updateEditStarDisplay(agent.rating || 0);
// Handle Quality Audit field
const qualityAuditCheckbox = document.getElementById('editQualityAuditStatus');
@ -1037,8 +1106,7 @@ async function updateAgent(e) {
agent_tags: document.getElementById('editAgentTags').value.split(',').map(s => s.trim()).filter(s => s),
agent_userbase: document.getElementById('editAgentUserbase').value.split(',').map(s => s.trim()).filter(s => s),
agent_capabilities: document.getElementById('editAgentCapabilities').value.split(',').map(s => s.trim()).filter(s => s),
discipline: document.getElementById('editAgentDiscipline').value || null,
rating: document.getElementById('editAgentRating').value ? parseFloat(document.getElementById('editAgentRating').value) : null
discipline: document.getElementById('editAgentDiscipline').value || null
};
// Add Quality Audit status and Risk Factor if user is admin
@ -1429,7 +1497,7 @@ function renderUsageChart(chartData) {
}
// Star rating helper functions
function getStarDisplay(rating) {
function getStarDisplay(rating, ratingCount) {
if (!rating) return '<span class="text-muted">Not yet rated</span>';
let stars = '';
for (let i = 1; i <= 5; i++) {
@ -1441,7 +1509,8 @@ function getStarDisplay(rating) {
stars += '<span class="text-muted">&#9734;</span>';
}
}
return stars + ` <small class="text-muted">(${rating.toFixed(1)})</small>`;
const countText = ratingCount ? ` (${ratingCount} rating${ratingCount !== 1 ? 's' : ''})` : '';
return stars + ` <small class="text-muted">${rating.toFixed(1)}${countText}</small>`;
}
function getClickableStars(currentRating, agentId) {
@ -1480,16 +1549,27 @@ function setupDetailStarListeners(agentId, currentRating) {
body: JSON.stringify({ rating: val })
});
if (response.ok) {
const data = await response.json();
currentRating = val;
stars.forEach(s => {
s.classList.toggle('text-warning', parseInt(s.dataset.value) <= val);
s.classList.toggle('text-muted', parseInt(s.dataset.value) > val);
});
// Update agents cache
// Update average display next to stars
const container = document.getElementById('detailRatingDisplay');
const existingAvg = container.querySelector('.rating-avg-text');
if (existingAvg) existingAvg.remove();
if (data.rating) {
const avgSpan = document.createElement('small');
avgSpan.className = 'text-muted rating-avg-text';
avgSpan.textContent = ` Avg: ${data.rating.toFixed(1)} (${data.rating_count} rating${data.rating_count !== 1 ? 's' : ''})`;
container.appendChild(avgSpan);
}
// Update agents cache with new average
const idx = allAgents.findIndex(a => a.agent_id === agentId);
if (idx !== -1) allAgents[idx].rating = val;
if (idx !== -1) { allAgents[idx].rating = data.rating; allAgents[idx].rating_count = data.rating_count; }
const myIdx = myAgents.findIndex(a => a.agent_id === agentId);
if (myIdx !== -1) myAgents[myIdx].rating = val;
if (myIdx !== -1) { myAgents[myIdx].rating = data.rating; myAgents[myIdx].rating_count = data.rating_count; }
} else {
showError('Failed to save rating');
}
@ -1500,46 +1580,6 @@ function setupDetailStarListeners(agentId, currentRating) {
});
}
// Edit modal star rating interaction
function setupEditStarRating() {
const container = document.getElementById('editStarRating');
if (!container) return;
const stars = container.querySelectorAll('.star-interactive');
const hiddenInput = document.getElementById('editAgentRating');
stars.forEach(star => {
star.addEventListener('mouseenter', function() {
const val = parseInt(this.dataset.value);
stars.forEach(s => {
s.style.color = parseInt(s.dataset.value) <= val ? '#ffc107' : '#dee2e6';
});
});
star.addEventListener('mouseleave', function() {
const current = parseInt(hiddenInput.value) || 0;
stars.forEach(s => {
s.style.color = parseInt(s.dataset.value) <= current ? '#ffc107' : '#dee2e6';
});
});
star.addEventListener('click', function() {
const val = parseInt(this.dataset.value);
hiddenInput.value = val;
stars.forEach(s => {
s.style.color = parseInt(s.dataset.value) <= val ? '#ffc107' : '#dee2e6';
});
});
});
}
function updateEditStarDisplay(rating) {
const container = document.getElementById('editStarRating');
if (!container) return;
const stars = container.querySelectorAll('.star-interactive');
const val = Math.round(rating) || 0;
stars.forEach(s => {
s.style.color = parseInt(s.dataset.value) <= val ? '#ffc107' : '#dee2e6';
});
}
function setupChartEventListeners(agentName) {
// Period toggle listeners
document.querySelectorAll('input[name="period"]').forEach(radio => {

1
templates/agent_register.html
View file

@ -110,6 +110,7 @@
<option value="Oversight including delivery">Oversight including delivery</option>
<option value="Optimization">Optimization</option>
<option value="Back Office including operations">Back Office including operations</option>
<option value="Pencil Agents">Pencil Agents</option>
</select>
</div>
<div class="col-md-6 mb-4">